-- Logs begin at Thu 2022-09-01 04:39:43 UTC, end at Thu 2022-09-01 06:03:01 UTC. -- Sep 01 05:11:32.310285 dv-cinder-852208-15-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 05:11:32.317614 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 05:11:32.319700 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 05:11:32 2022] *** Sep 01 05:11:32.319700 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 05:11:32.319700 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 05:11:32.319700 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: nodename: dv-cinder-852208-15-iscsi Sep 01 05:11:32.319700 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: machine: x86_64 Sep 01 05:11:32.319700 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: clock source: unix Sep 01 05:11:32.319700 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: pcre jit disabled Sep 01 05:11:32.319700 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: detected number of CPU cores: 4 Sep 01 05:11:32.319700 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: current working directory: / Sep 01 05:11:32.320369 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: detected binary path: /usr/bin/uwsgi-core Sep 01 05:11:32.320369 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: your processes number limit is 47680 Sep 01 05:11:32.320369 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: your memory page size is 4096 bytes Sep 01 05:11:32.320369 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: detected max file descriptor number: 2048 Sep 01 05:11:32.320369 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: lock engine: pthread robust mutexes Sep 01 05:11:32.321000 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: thunder lock: enabled Sep 01 05:11:32.322837 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 05:11:32.322837 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 05:11:32.361881 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: Python main interpreter initialized at 0x5616a369d9c0 Sep 01 05:11:32.361881 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: python threads support enabled Sep 01 05:11:32.361881 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: your server socket listen backlog is limited to 100 connections Sep 01 05:11:32.361881 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: your mercy for graceful operations on workers is 90 seconds Sep 01 05:11:32.362256 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 05:11:32.362333 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: *** Operational MODE: preforking *** Sep 01 05:11:32.362563 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: *** uWSGI is running in multiple interpreter mode *** Sep 01 05:11:32.362563 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: spawned uWSGI master process (pid: 81561) Sep 01 05:11:32.362778 dv-cinder-852208-15-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 05:11:32.366133 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: spawned uWSGI worker 1 (pid: 81574, cores: 1) Sep 01 05:11:32.368201 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: spawned uWSGI worker 2 (pid: 81577, cores: 1) Sep 01 05:11:32.368494 dv-cinder-852208-15-iscsi devstack@keystone.service[81561]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 05:11:36.907609 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 05:11:36.907609 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: from cryptography.utils import int_from_bytes Sep 01 05:11:36.908387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 05:11:36.908387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: from cryptography.utils import int_from_bytes Sep 01 05:11:36.932598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 05:11:36.932598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: from cryptography.utils import int_from_bytes Sep 01 05:11:36.933158 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 05:11:36.933158 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: from cryptography.utils import int_from_bytes Sep 01 05:11:36.962911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 05:11:36.963103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 05:11:36.963295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] command line args: [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 05:11:36.963561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 05:11:36.963736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ================================================================================ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 05:11:36.963986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] admin_token = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.964229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config_dir = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.964414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.964592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config_source = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.964768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] debug = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.964962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.966369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.966369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.966369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.966369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.966369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.966741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_config_append = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.966741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.966741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_dir = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.966955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.967144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.967391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.967633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.967879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.968057 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.968234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.968477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.968721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.968953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.969183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.969413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.969670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.969913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.970154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.970424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.970916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.971095 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] publish_errors = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.971327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.971555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.971731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.971974 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.972209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.972439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] standard_threads = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.972665 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.972906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.973106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.973417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_journal = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.973787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_json = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.974006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_stderr = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.974186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_syslog = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.974426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.974687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.974929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.975192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.975431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.975681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.975919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.976170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.976405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.976678 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.976898 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.977167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.977434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.977676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.977912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.978160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.978407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.978643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.978897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.979149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.979393 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.979655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.979881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.980137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.980386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.980633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.980863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.981092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.981249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 05:11:36.981342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.981445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 05:11:36.981615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.981692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] command line args: [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 05:11:36.981799 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.981920 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 05:11:36.982048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.982144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ================================================================================ {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 05:11:36.982275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.982353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] admin_token = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.982503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.982756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] config_dir = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.982834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.982917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.982995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] config_source = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.983071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.983142 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] debug = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.983211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.983317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.983421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.983554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.983636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.983732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.983869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.983982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.984102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.984231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.984313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.984461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.984545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.984691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_config_append = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.984770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.984847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.984963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_dir = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.985040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.985139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_file = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.985228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.985372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.985476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.985652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.985730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.985889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.985980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.986119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.986208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.986287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.986358 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.986461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.986571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.986706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.986816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.986951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.987029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.987234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.987329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.987442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.987520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.987635 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.987719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.987865 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.987944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.988109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.988191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.988352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.988428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.989116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.989205 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.989507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.989661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.990833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] publish_errors = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.990952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.991038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.991116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.991265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.991265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.991406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.991406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.991540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.991610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.991684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.991755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.991833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] standard_threads = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.991905 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.991974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.992042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.992111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.992111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.992245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.992322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] use_journal = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.992396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.992466 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] use_json = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.992535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.992604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] use_stderr = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.992604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] use_syslog = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.992736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.992810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:11:36.992881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.992950 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.993059 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.993164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.993261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.993394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.993513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.993659 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.993761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.994013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.994136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.994136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.994340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.994490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.994578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.994659 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.994776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.994894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.994970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.995086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.995161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.995284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.995360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.995476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.995595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.995738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.995858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.995982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.996062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.996229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.996319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.996468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.996551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.996706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.996790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.996944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.997020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.997180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.997258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.997415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.997525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.997664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.997743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.997895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.998014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.998124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.998211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.998354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.998431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.998587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.998668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.998815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.998902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.999054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.999136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.999289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.999367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.999535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.999621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.999771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:36.999855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.000025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.000104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.000276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.000360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.000512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.000589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.000740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.external = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.000822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.000970 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.001058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.001232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.001310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.001499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.001582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.001733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.password = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.001812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.001962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.token = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.002039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.002199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.002312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.002461 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.002536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.002674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.002752 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.002904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.002981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.003150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.003238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.003387 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.003465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.003614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.003692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.003851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.003933 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.004088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.004169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.004962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.005050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.005124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.005399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.005558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.005751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.005884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.005962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.006041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.006152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.006271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.006391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.006523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.006630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.006780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.006866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.007019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.007099 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.007273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.007390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.007548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.007634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.007786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.007870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.008026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.008104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.008258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.008346 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.008524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.008656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.008763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.008863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.008996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.009076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.009237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.009316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.009516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.009600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.009727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.009804 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.009976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.010056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.010216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.010320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.010459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.010537 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.010694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.010772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.010935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.011013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.011171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.011248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.011416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.011502 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.011648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.011731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.011882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.011978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.012127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.012215 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.012359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.012436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.012598 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.012676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.012832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.012921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.013053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.013183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.013286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.013366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.013541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.017565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.017693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.017781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.017781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.017913 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.017991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018216 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018601 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018814 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.018971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019314 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019461 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.019967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020896 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.020974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021153 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.external = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.password = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.021948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.token = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022479 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.022997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023359 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023430 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023654 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.023933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024447 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.024935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.025101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.025180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.025335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.025515 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.025611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.025783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.025862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.026023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.026107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.026255 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.026342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.026485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.026564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.026730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.026808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.026959 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.027048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.027203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.027281 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.027441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.027558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.027682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.027769 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.027911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.027989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.028157 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.028258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.028393 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.028509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.028635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.028776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.029021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.029098 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.029273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.029535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.029617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.029775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.030006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.030084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.030253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.030339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.030487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.030566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.030732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.030811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.030977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.031062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.031224 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.031302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.031456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.031533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.031691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.031777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.031924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.032003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.032170 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.032280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.032420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.032532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.032669 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.032767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.032924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.033003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.033172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.033259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.033445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.033548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.033706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.033785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.033944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.034031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.034177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.034283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.034413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.034493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.034645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.034732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.034880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.034959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.035119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.035198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.035360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.036274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.036368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.036442 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.036514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.036674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.036764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.036982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.037086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.037240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.037318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.037524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.037662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.037780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.037859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.038029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.038109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.038278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.038369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.038509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.038587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.038750 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.038828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.038992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.039081 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.039237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.039316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.039477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.039556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.039716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.039803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.039959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.040037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.040205 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.040283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.040440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.040618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.040700 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.040856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.040935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.041105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.041185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.041358 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.041533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.041612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.041683 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.041846 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.041926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.042079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.042160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.042319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.042404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.042579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.042710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.042797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.042872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.043049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.043129 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.043296 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.043373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.043527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.043607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.043778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.043856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.044024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.044101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.044262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.044343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.044493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.044570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.044727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.044813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.044954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.045033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.045220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.045300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.045471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048562 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.048969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049173 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049314 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.049992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.050922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.051001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.051095 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.051719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.051787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.051847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.051919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.051999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052078 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.052999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.053135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.053214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.053285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.053520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.053604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.053717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.053807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.053953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.054031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] role.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.054199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.054278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] role.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.054431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.054516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.054671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.054749 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.054907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.054986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.055066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 05:11:37.055141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.055388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.055637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.055886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.056133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.056383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.056619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.056869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.057119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.057367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.057667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.058123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.058367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.058621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.058874 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.059125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.059376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.059612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.059868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.060119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.060360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.060592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.060825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.061059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.061312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.061598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.061855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.062095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.062415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.062661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.062902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.063143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.063395 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.063636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.063894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.064143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.064382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.064627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.064871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.065119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.065357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.065624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.065915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.066172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.066446 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.066680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.066981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.067251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.067516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.connection = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.067757 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.068006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.068244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.068486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.068861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.068998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.069167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.069405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.069671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.069908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.070152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.075122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.075122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.075122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.075122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.075122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.075122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.075122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:11:37.075122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 05:11:37.085100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.085328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.085612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.093930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.096984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.097217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.097439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.106807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.127152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.127383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.134632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.134868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.186018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.186194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.239893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.240138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.250894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.251132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.412716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.412958 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.413152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.413374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.413617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.413837 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.414253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.414488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.414711 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.414929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.415144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.415356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.437425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.437787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.437875 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.438022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.438106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.438242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.438327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.438459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.438543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.438674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.438765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.438881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.438964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.439090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.450540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.450902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.463924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.464157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.464280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.464367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.464514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.464614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.464739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.464824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.464960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.465045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.465165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.465250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.465381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.465627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:11:37.482179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.482385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 05:11:37.482594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.482806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 05:11:37.482960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.483151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.483235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 05:11:37.483354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 05:11:37.483438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.483633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.483716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.483898 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.483980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.484148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.484231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.484471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.494165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.494396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.494665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.494780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.494946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.495064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.495337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.495620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.501944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.502754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.505571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.505856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.506140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.506237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.506422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.506532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.506698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.506819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.506955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.507104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.507333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.507416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.507606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.507697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.507876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.507959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.508149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.508231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.508417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.508507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.508709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.508959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.530921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.531995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.532379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.532664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.533512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.533805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.537821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.538106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.538387 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.539018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.539438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.539843 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.544731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.545025 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.548383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.548680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.549123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.549413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.549794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.550074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.550345 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.550615 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.550880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.551678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.553225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.553548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.553865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.554165 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.554443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.556509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.556800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.557081 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.569811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.570093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.574147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.574395 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.574480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.577782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.578078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.578584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.581725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.582030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.582324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.585685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.586554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.588812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.589110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.589394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.589715 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.589801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.590096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.590554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.592821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.593123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.593412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.593725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.594017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.594403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.606416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.606704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.607036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.607308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.610288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.610579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.610845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.611108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.615086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.616974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.618432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.618548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.618825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.620089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.621602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.621689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.621897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.621994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.622179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.622470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.622762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.624563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.624852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.625148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.625432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.625745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.634677 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.634966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.635247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.635528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.635809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.636086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.636378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.637731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.638021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.638305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.638591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.638877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.639155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.639452 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.651810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.654737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.654895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.657621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.657966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.659173 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.660755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.660985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.661069 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.661320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.662490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.663982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.664266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.664565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.668152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.669754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.671562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.672634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.672926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.673054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.673211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.673517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.675946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.676914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.677201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.677508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.682433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.682726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.683001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.685472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.685772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.686054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.690124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.690406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.690681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.690952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.691223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.691508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.691872 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.692136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.692535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.692805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.693085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.693259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:11:37.693344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.693579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.693693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.693878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.693960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.694161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.694438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.694718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.694995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.695272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.695552 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.695829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.696101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.696362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.696627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.696907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:11:37.729347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5616a369d9c0 pid: 81577 (default app) Sep 01 05:11:37.731866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5616a369d9c0 pid: 81574 (default app) Sep 01 05:11:37.732726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eb171-3675-4e6b-b73a-38a1f51d031e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.733010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eb171-3675-4e6b-b73a-38a1f51d031e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.733279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eb171-3675-4e6b-b73a-38a1f51d031e None None] PATH_INFO: `/v3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.737545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:45.167219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cae802-114f-4ae3-90dd-0be913ae182e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:45.167745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cae802-114f-4ae3-90dd-0be913ae182e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:45.167827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cae802-114f-4ae3-90dd-0be913ae182e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:45.172093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:11:45.177337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:45.177788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:45.177951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:45.251644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: INFO dbcounter [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] Registered counter for database keystone Sep 01 05:11:45.270783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG oslo_db.sqlalchemy.engines [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81577) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 05:11:45.277145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writer thread running {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 05:11:45.458256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG passlib.handlers.bcrypt [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81577) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 05:11:45.458567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG passlib.handlers.bcrypt [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81577) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 05:11:45.516822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:45.518474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:45.648440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 05:11:46.151671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cae802-114f-4ae3-90dd-0be913ae182e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:46.221218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: INFO dbcounter [None req-c2cae802-114f-4ae3-90dd-0be913ae182e None None] Registered counter for database keystone Sep 01 05:11:46.242307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG oslo_db.sqlalchemy.engines [None req-c2cae802-114f-4ae3-90dd-0be913ae182e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81574) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 05:11:46.248383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writer thread running {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 05:11:46.367016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cae802-114f-4ae3-90dd-0be913ae182e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:46.371243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a268544-262a-422e-a2d8-8de35910d542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:46.371859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a268544-262a-422e-a2d8-8de35910d542 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:46.372134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a268544-262a-422e-a2d8-8de35910d542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:46.372403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a268544-262a-422e-a2d8-8de35910d542 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:46.380398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a268544-262a-422e-a2d8-8de35910d542 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:46.432247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1a268544-262a-422e-a2d8-8de35910d542 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 05:11:46.432531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a268544-262a-422e-a2d8-8de35910d542 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:46.441928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Sep 1 05:11:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:11:46.448278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:46.465771 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f997c4-d2bd-44ec-832f-b69999ab557c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:46.470215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adf099-b1f2-4859-bd1e-c41987e77beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:46.471322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adf099-b1f2-4859-bd1e-c41987e77beb admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:46.471602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adf099-b1f2-4859-bd1e-c41987e77beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:46.471870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adf099-b1f2-4859-bd1e-c41987e77beb admin admin] PATH_INFO: `/v3/projects/61f9dee9572644e580e35da89b6265e9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:46.473426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adf099-b1f2-4859-bd1e-c41987e77beb admin admin] RBAC: Authorizing `identity:get_project(project_id=61f9dee9572644e580e35da89b6265e9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:46.528255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: WARNING keystone.common.rbac_enforcer.enforcer [None req-00adf099-b1f2-4859-bd1e-c41987e77beb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 05:11:46.528592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adf099-b1f2-4859-bd1e-c41987e77beb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:46.533204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Thu Sep 1 05:11:46 2022] GET /identity/v3/projects/61f9dee9572644e580e35da89b6265e9 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:48.428046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3d1ce-b859-4576-baad-3df8ad297af6 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.428573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3d1ce-b859-4576-baad-3df8ad297af6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.428765 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3d1ce-b859-4576-baad-3df8ad297af6 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.432881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:48.438183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a1f78-836c-41cb-842f-9367d1d30638 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.438544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a1f78-836c-41cb-842f-9367d1d30638 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.438742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a1f78-836c-41cb-842f-9367d1d30638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.459955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2a0a1f78-836c-41cb-842f-9367d1d30638 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:48.461501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2a0a1f78-836c-41cb-842f-9367d1d30638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:48.469352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 05:11:48.739209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3d1ce-b859-4576-baad-3df8ad297af6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.752155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3d1ce-b859-4576-baad-3df8ad297af6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.757056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacddd9-3c3c-40d5-84b8-68ad69e1a432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.757681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacddd9-3c3c-40d5-84b8-68ad69e1a432 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.757963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacddd9-3c3c-40d5-84b8-68ad69e1a432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.758234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacddd9-3c3c-40d5-84b8-68ad69e1a432 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.765863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacddd9-3c3c-40d5-84b8-68ad69e1a432 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.767633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacddd9-3c3c-40d5-84b8-68ad69e1a432 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.776732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Sep 1 05:11:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:11:48.782109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a1f78-836c-41cb-842f-9367d1d30638 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.791490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a1f78-836c-41cb-842f-9367d1d30638 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.795829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094a5a6-185e-40f0-8a1e-68a6ed0ba9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.796442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094a5a6-185e-40f0-8a1e-68a6ed0ba9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.796719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094a5a6-185e-40f0-8a1e-68a6ed0ba9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.796989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094a5a6-185e-40f0-8a1e-68a6ed0ba9f2 admin admin] PATH_INFO: `/v3/users/9d4d38ef7ecd40a380aba27a2b564d42` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.798304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094a5a6-185e-40f0-8a1e-68a6ed0ba9f2 admin admin] RBAC: Authorizing `identity:get_user(user_id=9d4d38ef7ecd40a380aba27a2b564d42)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.800111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094a5a6-185e-40f0-8a1e-68a6ed0ba9f2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.801635 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Thu Sep 1 05:11:48 2022] GET /identity/v3/users/9d4d38ef7ecd40a380aba27a2b564d42 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:50.793895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cf5a1-2999-414c-8d70-d2cfe424ee6f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.794402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cf5a1-2999-414c-8d70-d2cfe424ee6f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.794885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cf5a1-2999-414c-8d70-d2cfe424ee6f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.797305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:11:50.803667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fade2-27b9-4573-8fee-d7c2398fc1f7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.804075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fade2-27b9-4573-8fee-d7c2398fc1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.804226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fade2-27b9-4573-8fee-d7c2398fc1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.822975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-203fade2-27b9-4573-8fee-d7c2398fc1f7 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:50.824738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-203fade2-27b9-4573-8fee-d7c2398fc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:50.825538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276a329-b48a-4e92-9c9e-3d4d6927e3ae None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.825828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276a329-b48a-4e92-9c9e-3d4d6927e3ae None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.826102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276a329-b48a-4e92-9c9e-3d4d6927e3ae None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.827404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:11:50.832339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fddb0a-1aed-40bc-8096-1213ea320dd2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.832617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fddb0a-1aed-40bc-8096-1213ea320dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.833301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fddb0a-1aed-40bc-8096-1213ea320dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.876265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG passlib.handlers.bcrypt [None req-81fddb0a-1aed-40bc-8096-1213ea320dd2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81574) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 05:11:50.876661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG passlib.handlers.bcrypt [None req-81fddb0a-1aed-40bc-8096-1213ea320dd2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81574) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 05:11:50.908391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:50.926479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-81fddb0a-1aed-40bc-8096-1213ea320dd2 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:50.928397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-81fddb0a-1aed-40bc-8096-1213ea320dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:51.028271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:11:51.175603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fade2-27b9-4573-8fee-d7c2398fc1f7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.199909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.286396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fade2-27b9-4573-8fee-d7c2398fc1f7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.310897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fddb0a-1aed-40bc-8096-1213ea320dd2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.320247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6af25d-2df1-495c-a18b-d3e1324a25e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.320896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6af25d-2df1-495c-a18b-d3e1324a25e0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.321178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6af25d-2df1-495c-a18b-d3e1324a25e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.321466 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6af25d-2df1-495c-a18b-d3e1324a25e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.333664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6af25d-2df1-495c-a18b-d3e1324a25e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-6e6af25d-2df1-495c-a18b-d3e1324a25e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:51.342928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:51.343931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:11:51.344935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:51.349799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6af25d-2df1-495c-a18b-d3e1324a25e0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.416967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fddb0a-1aed-40bc-8096-1213ea320dd2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.419262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.431130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.449596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6af25d-2df1-495c-a18b-d3e1324a25e0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.453963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373e887-ac7c-49ee-bd36-00bd13f8b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.454822 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373e887-ac7c-49ee-bd36-00bd13f8b553 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.455162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373e887-ac7c-49ee-bd36-00bd13f8b553 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.455432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373e887-ac7c-49ee-bd36-00bd13f8b553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.485287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155fc64-929a-4ea7-a80d-9dac824af9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.485921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155fc64-929a-4ea7-a80d-9dac824af9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.486203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155fc64-929a-4ea7-a80d-9dac824af9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.486478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155fc64-929a-4ea7-a80d-9dac824af9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.487140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155fc64-929a-4ea7-a80d-9dac824af9a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:51.487444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155fc64-929a-4ea7-a80d-9dac824af9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.488835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155fc64-929a-4ea7-a80d-9dac824af9a9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.501374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:51.506992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155fc64-929a-4ea7-a80d-9dac824af9a9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.524680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373e887-ac7c-49ee-bd36-00bd13f8b553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.526311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373e887-ac7c-49ee-bd36-00bd13f8b553 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.609653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155fc64-929a-4ea7-a80d-9dac824af9a9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.624674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:11:51.630381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373e887-ac7c-49ee-bd36-00bd13f8b553 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.633198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.647929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b9cab-f692-4d75-a501-d08939af4579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.648552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b9cab-f692-4d75-a501-d08939af4579 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.648827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b9cab-f692-4d75-a501-d08939af4579 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.649695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b9cab-f692-4d75-a501-d08939af4579 None admin] PATH_INFO: `/v3/users/9d4d38ef7ecd40a380aba27a2b564d42` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.650294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b9cab-f692-4d75-a501-d08939af4579 None admin] RBAC: Authorizing `identity:get_user(user_id=9d4d38ef7ecd40a380aba27a2b564d42)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.651756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b9cab-f692-4d75-a501-d08939af4579 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.653303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/users/9d4d38ef7ecd40a380aba27a2b564d42 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:51.658730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b9cab-f692-4d75-a501-d08939af4579 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.673733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.735114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373e887-ac7c-49ee-bd36-00bd13f8b553 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.760949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b9cab-f692-4d75-a501-d08939af4579 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.768872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debc032-ee68-4b3f-917f-ea046ee76da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.769762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debc032-ee68-4b3f-917f-ea046ee76da6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.770047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debc032-ee68-4b3f-917f-ea046ee76da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.770352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debc032-ee68-4b3f-917f-ea046ee76da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.790021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debc032-ee68-4b3f-917f-ea046ee76da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.791491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debc032-ee68-4b3f-917f-ea046ee76da6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.797133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e8b3e-1209-4f46-a236-c2b9139cc40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.797801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e8b3e-1209-4f46-a236-c2b9139cc40c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.798082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e8b3e-1209-4f46-a236-c2b9139cc40c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.798357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e8b3e-1209-4f46-a236-c2b9139cc40c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.799482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e8b3e-1209-4f46-a236-c2b9139cc40c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.800924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e8b3e-1209-4f46-a236-c2b9139cc40c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.802515 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:51.808717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e8b3e-1209-4f46-a236-c2b9139cc40c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3debc032-ee68-4b3f-917f-ea046ee76da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:51.810307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:51.811472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:51.812284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:51.812284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:51.812284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:11:51.812284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:51.816985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debc032-ee68-4b3f-917f-ea046ee76da6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.874759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.892576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.912651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e8b3e-1209-4f46-a236-c2b9139cc40c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.922116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debc032-ee68-4b3f-917f-ea046ee76da6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.947107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a6146-7e05-4bd8-aaa2-a13c01115f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.947734 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a6146-7e05-4bd8-aaa2-a13c01115f7a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.948010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a6146-7e05-4bd8-aaa2-a13c01115f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.948282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a6146-7e05-4bd8-aaa2-a13c01115f7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.948997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a6146-7e05-4bd8-aaa2-a13c01115f7a None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.domain.id=default, user.id=9d4d38ef7ecd40a380aba27a2b564d42) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:51.949300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a6146-7e05-4bd8-aaa2-a13c01115f7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.951031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a6146-7e05-4bd8-aaa2-a13c01115f7a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.955809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a620c0-e756-486a-9a31-4f6117136f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.956421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a620c0-e756-486a-9a31-4f6117136f6e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.956705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a620c0-e756-486a-9a31-4f6117136f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.956973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a620c0-e756-486a-9a31-4f6117136f6e None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.957569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/role_assignments?user.id=9d4d38ef7ecd40a380aba27a2b564d42&scope.domain.id=default&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:51.957669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a620c0-e756-486a-9a31-4f6117136f6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:51.957930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a620c0-e756-486a-9a31-4f6117136f6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.960034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a620c0-e756-486a-9a31-4f6117136f6e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.982862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:54.236200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d52212-262a-4628-b3ff-bdbfc4448118 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.236200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d52212-262a-4628-b3ff-bdbfc4448118 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.236200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d52212-262a-4628-b3ff-bdbfc4448118 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.236200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:54.245700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd9a63-b5a6-4b96-b814-25a2e1d81131 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.245700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd9a63-b5a6-4b96-b814-25a2e1d81131 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.245700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd9a63-b5a6-4b96-b814-25a2e1d81131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.275019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4fdd9a63-b5a6-4b96-b814-25a2e1d81131 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:54.276226 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4fdd9a63-b5a6-4b96-b814-25a2e1d81131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:54.350352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d26c8-aecc-4d27-a75b-4947b1ee69a8 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.350987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d26c8-aecc-4d27-a75b-4947b1ee69a8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.351488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d26c8-aecc-4d27-a75b-4947b1ee69a8 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.353263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 13/23] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:54.359707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd52e76-b0be-416f-9e8f-8f8ab1128fb2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.360284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd52e76-b0be-416f-9e8f-8f8ab1128fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.360765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd52e76-b0be-416f-9e8f-8f8ab1128fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.374844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 11/24] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:54.391028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2bd52e76-b0be-416f-9e8f-8f8ab1128fb2 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:54.392223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2bd52e76-b0be-416f-9e8f-8f8ab1128fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:54.482554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 14/25] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:11:54.908537 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cd875-8c20-4b42-bea6-c8c44721109a None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.909353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cd875-8c20-4b42-bea6-c8c44721109a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.909864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cd875-8c20-4b42-bea6-c8c44721109a None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.911411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 12/26] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:54.913357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd52e76-b0be-416f-9e8f-8f8ab1128fb2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:54.922902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a477c-1dce-4dbd-bbf3-18f54ca44e89 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.923463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a477c-1dce-4dbd-bbf3-18f54ca44e89 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.923954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a477c-1dce-4dbd-bbf3-18f54ca44e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.943116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d72a477c-1dce-4dbd-bbf3-18f54ca44e89 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:54.944973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d72a477c-1dce-4dbd-bbf3-18f54ca44e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:54.992812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:55.018714 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 13/27] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:55.020336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a059eb3-0dc7-42f2-ab85-da2c352a89d1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.020771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a059eb3-0dc7-42f2-ab85-da2c352a89d1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.021879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a059eb3-0dc7-42f2-ab85-da2c352a89d1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.023516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 14/28] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:55.024396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a059eb3-0dc7-42f2-ab85-da2c352a89d1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:55.044949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:55.067797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd52e76-b0be-416f-9e8f-8f8ab1128fb2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:55.111373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e66e59-aed8-4260-8df1-d77c85c1733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:55.112010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e66e59-aed8-4260-8df1-d77c85c1733f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.112290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e66e59-aed8-4260-8df1-d77c85c1733f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.112557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e66e59-aed8-4260-8df1-d77c85c1733f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.140344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e66e59-aed8-4260-8df1-d77c85c1733f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-89e66e59-aed8-4260-8df1-d77c85c1733f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:55.150123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:55.151085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:11:55.151953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 15/29] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:55.152611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b009-e6e4-4c38-929b-b8ae6eaf16a5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.153550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b009-e6e4-4c38-929b-b8ae6eaf16a5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.153550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b009-e6e4-4c38-929b-b8ae6eaf16a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.190222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2cf9b009-e6e4-4c38-929b-b8ae6eaf16a5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:55.191485 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2cf9b009-e6e4-4c38-929b-b8ae6eaf16a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:55.222061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a059eb3-0dc7-42f2-ab85-da2c352a89d1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:55.306370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e22ea-351a-4b64-9635-c9e38a3ef1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:55.308105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e22ea-351a-4b64-9635-c9e38a3ef1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.309752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e22ea-351a-4b64-9635-c9e38a3ef1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.310044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e22ea-351a-4b64-9635-c9e38a3ef1f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.332683 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e22ea-351a-4b64-9635-c9e38a3ef1f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:55.336228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e22ea-351a-4b64-9635-c9e38a3ef1f2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:55.341502 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 16/30] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:55.342360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9b009-e6e4-4c38-929b-b8ae6eaf16a5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5f7e22ea-351a-4b64-9635-c9e38a3ef1f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:55.363093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:55.365443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:55.367554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:55.367554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:55.367554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:11:55.371511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 15/31] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:55.378560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e22ea-351a-4b64-9635-c9e38a3ef1f2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:55.434910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:55.469831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:55.543064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9b009-e6e4-4c38-929b-b8ae6eaf16a5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:55.562193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e22ea-351a-4b64-9635-c9e38a3ef1f2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:55.583247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f67108-69f5-438f-ac6c-fb883b5e877e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:55.584641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f67108-69f5-438f-ac6c-fb883b5e877e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.585061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f67108-69f5-438f-ac6c-fb883b5e877e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.585929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f67108-69f5-438f-ac6c-fb883b5e877e None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.587046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f67108-69f5-438f-ac6c-fb883b5e877e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:55.587563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f67108-69f5-438f-ac6c-fb883b5e877e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:55.589672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f67108-69f5-438f-ac6c-fb883b5e877e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:55.600267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd45e8-f6fa-4f7a-a60d-d674d1f81ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:55.600964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd45e8-f6fa-4f7a-a60d-d674d1f81ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.601313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd45e8-f6fa-4f7a-a60d-d674d1f81ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.601854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 17/32] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:55.602174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd45e8-f6fa-4f7a-a60d-d674d1f81ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.602832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd45e8-f6fa-4f7a-a60d-d674d1f81ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:55.603198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd45e8-f6fa-4f7a-a60d-d674d1f81ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:55.603379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f67108-69f5-438f-ac6c-fb883b5e877e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:55.604751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd45e8-f6fa-4f7a-a60d-d674d1f81ab4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:55.620351 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 16/33] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:55.622318 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c445a-af7d-4354-be43-08022f459d8d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.622788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c445a-af7d-4354-be43-08022f459d8d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.623237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c445a-af7d-4354-be43-08022f459d8d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.624935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 17/34] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:55.625960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c445a-af7d-4354-be43-08022f459d8d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:55.722553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f67108-69f5-438f-ac6c-fb883b5e877e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:55.725218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:55.738504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c445a-af7d-4354-be43-08022f459d8d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:55.756651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:55.765099 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8445b2-0832-47e0-ba7a-a97313925518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:55.765843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8445b2-0832-47e0-ba7a-a97313925518 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.766075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8445b2-0832-47e0-ba7a-a97313925518 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.766349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8445b2-0832-47e0-ba7a-a97313925518 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.767023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8445b2-0832-47e0-ba7a-a97313925518 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:55.768417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8445b2-0832-47e0-ba7a-a97313925518 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:55.777876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586b17e-fed5-479a-93e7-8a8b3ddc5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:55.778529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586b17e-fed5-479a-93e7-8a8b3ddc5145 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.778813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586b17e-fed5-479a-93e7-8a8b3ddc5145 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.779088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586b17e-fed5-479a-93e7-8a8b3ddc5145 None admin] PATH_INFO: `/v3/users/9d4d38ef7ecd40a380aba27a2b564d42` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.780562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586b17e-fed5-479a-93e7-8a8b3ddc5145 None admin] RBAC: Authorizing `identity:get_user(user_id=9d4d38ef7ecd40a380aba27a2b564d42)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:55.782140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586b17e-fed5-479a-93e7-8a8b3ddc5145 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:55.783735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 18/35] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:11:55 2022] GET /identity/v3/users/9d4d38ef7ecd40a380aba27a2b564d42 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:55.784481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586b17e-fed5-479a-93e7-8a8b3ddc5145 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:55.890710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586b17e-fed5-479a-93e7-8a8b3ddc5145 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:55.924968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae8894-cbce-4b73-8542-28c12cf95611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:55.925611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae8894-cbce-4b73-8542-28c12cf95611 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.925896 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae8894-cbce-4b73-8542-28c12cf95611 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.926168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae8894-cbce-4b73-8542-28c12cf95611 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.926831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae8894-cbce-4b73-8542-28c12cf95611 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:55.928405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae8894-cbce-4b73-8542-28c12cf95611 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:55.962488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 18/36] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 1 05:11:55 2022] POST /identity/v3/roles => generated 225 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:55.963938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a22118-cbda-41dc-a0e0-677aca21358a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.964231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a22118-cbda-41dc-a0e0-677aca21358a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.964530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a22118-cbda-41dc-a0e0-677aca21358a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.985102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d6a22118-cbda-41dc-a0e0-677aca21358a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:55.986808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d6a22118-cbda-41dc-a0e0-677aca21358a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:56.086131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 19/37] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:56.086131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a22118-cbda-41dc-a0e0-677aca21358a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:56.155145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 19/38] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:11:55 2022] POST /identity/v3/projects => generated 301 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:56.156500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae8894-cbce-4b73-8542-28c12cf95611 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:56.169216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:56.220133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a22118-cbda-41dc-a0e0-677aca21358a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:56.251350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:56.274812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21b34e-a56b-4e76-b73c-3b7a92a594a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:56.275444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21b34e-a56b-4e76-b73c-3b7a92a594a6 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.275744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21b34e-a56b-4e76-b73c-3b7a92a594a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.282202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21b34e-a56b-4e76-b73c-3b7a92a594a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.282202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21b34e-a56b-4e76-b73c-3b7a92a594a6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:56.282202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21b34e-a56b-4e76-b73c-3b7a92a594a6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:56.299153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae8894-cbce-4b73-8542-28c12cf95611 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:56.340749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eee772-988c-44fb-8b13-0f43e6ecb23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:56.341386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eee772-988c-44fb-8b13-0f43e6ecb23b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.341739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eee772-988c-44fb-8b13-0f43e6ecb23b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.342013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eee772-988c-44fb-8b13-0f43e6ecb23b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.343230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eee772-988c-44fb-8b13-0f43e6ecb23b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:56.344779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eee772-988c-44fb-8b13-0f43e6ecb23b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:56.346563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 20/39] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:56.347631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa068e-6632-4e7b-b26f-5c7c921a9510 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.353331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa068e-6632-4e7b-b26f-5c7c921a9510 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.353331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa068e-6632-4e7b-b26f-5c7c921a9510 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.353331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 21/40] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:56.353331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa068e-6632-4e7b-b26f-5c7c921a9510 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:56.359819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 20/41] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 1 05:11:56 2022] POST /identity/v3/roles => generated 219 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:56.361122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf1c16-129a-427a-afb2-4900be4404ff None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.361410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf1c16-129a-427a-afb2-4900be4404ff None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.361716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf1c16-129a-427a-afb2-4900be4404ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.387586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-abcf1c16-129a-427a-afb2-4900be4404ff None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:56.388861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-abcf1c16-129a-427a-afb2-4900be4404ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:56.471961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:56.479472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 21/42] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:11:56.485698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf1c16-129a-427a-afb2-4900be4404ff None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:56.504452 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa068e-6632-4e7b-b26f-5c7c921a9510 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:56.551695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11f516-45ed-4e19-954f-e19c2a91290f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:56.552416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11f516-45ed-4e19-954f-e19c2a91290f None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.552841 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11f516-45ed-4e19-954f-e19c2a91290f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.553180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11f516-45ed-4e19-954f-e19c2a91290f None admin] PATH_INFO: `/v3/domains/default/users/9d4d38ef7ecd40a380aba27a2b564d42/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.556238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11f516-45ed-4e19-954f-e19c2a91290f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9d4d38ef7ecd40a380aba27a2b564d42, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:56.557729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11f516-45ed-4e19-954f-e19c2a91290f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:56.611460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf1c16-129a-427a-afb2-4900be4404ff None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:56.629691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 22/43] 10.222.0.42 () {58 vars in 1361 bytes} [Thu Sep 1 05:11:56 2022] PUT /identity/v3/domains/default/users/9d4d38ef7ecd40a380aba27a2b564d42/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:11:56.729520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:56.752009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11f516-45ed-4e19-954f-e19c2a91290f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:56.760112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc0e57-649a-470c-b9b5-57fd8af7d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:56.760745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc0e57-649a-470c-b9b5-57fd8af7d116 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.761030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc0e57-649a-470c-b9b5-57fd8af7d116 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.761308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc0e57-649a-470c-b9b5-57fd8af7d116 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.762097 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc0e57-649a-470c-b9b5-57fd8af7d116 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:56.763529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc0e57-649a-470c-b9b5-57fd8af7d116 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:56.812056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 22/44] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 1 05:11:56 2022] POST /identity/v3/roles => generated 223 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:56.867567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11f516-45ed-4e19-954f-e19c2a91290f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:56.905364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c8e24-0ba4-4950-8b00-91de81212180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:56.906150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c8e24-0ba4-4950-8b00-91de81212180 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.906538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c8e24-0ba4-4950-8b00-91de81212180 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.906889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c8e24-0ba4-4950-8b00-91de81212180 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.908124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c8e24-0ba4-4950-8b00-91de81212180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:56.909698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c8e24-0ba4-4950-8b00-91de81212180 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:56.911594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 23/45] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:56.917727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc0e57-649a-470c-b9b5-57fd8af7d116 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:57.026950 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc0e57-649a-470c-b9b5-57fd8af7d116 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:57.061018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03021663-09da-4287-96fd-3582e1fa3331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:57.061659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03021663-09da-4287-96fd-3582e1fa3331 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:57.061943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03021663-09da-4287-96fd-3582e1fa3331 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:57.062227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03021663-09da-4287-96fd-3582e1fa3331 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:57.062886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03021663-09da-4287-96fd-3582e1fa3331 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:57.064329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03021663-09da-4287-96fd-3582e1fa3331 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:57.158648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:11:56 2022] POST /identity/v3/projects => generated 312 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:58.561592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f93a1-3516-4f45-8b21-c4e4f5a70990 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:58.562146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f93a1-3516-4f45-8b21-c4e4f5a70990 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:58.562146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f93a1-3516-4f45-8b21-c4e4f5a70990 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:58.563472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 24/47] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:58.569184 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385860e7-356b-4600-b8dc-b3db2940037a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:58.569629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385860e7-356b-4600-b8dc-b3db2940037a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:58.569780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385860e7-356b-4600-b8dc-b3db2940037a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:58.588319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-385860e7-356b-4600-b8dc-b3db2940037a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:58.589549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-385860e7-356b-4600-b8dc-b3db2940037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:58.602769 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:58.656614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 24/48] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:58.924447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f93a1-3516-4f45-8b21-c4e4f5a70990 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:58.929138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:59.029407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f93a1-3516-4f45-8b21-c4e4f5a70990 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:59.063196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26438b66-0a35-4447-9d36-e1394a26b05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:59.063835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26438b66-0a35-4447-9d36-e1394a26b05f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.064120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26438b66-0a35-4447-9d36-e1394a26b05f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.064396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26438b66-0a35-4447-9d36-e1394a26b05f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.076395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26438b66-0a35-4447-9d36-e1394a26b05f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-26438b66-0a35-4447-9d36-e1394a26b05f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:59.082323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:59.083153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:11:59.083940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 25/49] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:59.089374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385860e7-356b-4600-b8dc-b3db2940037a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:59.095708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298e7ba-a9af-46b6-81bb-018595693552 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.095998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298e7ba-a9af-46b6-81bb-018595693552 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.096280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298e7ba-a9af-46b6-81bb-018595693552 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.097622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 26/50] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:11:59.103177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3d50d-45ea-4d15-b50d-205e1d79f224 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.103469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3d50d-45ea-4d15-b50d-205e1d79f224 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.103751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3d50d-45ea-4d15-b50d-205e1d79f224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.122698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6eb3d50d-45ea-4d15-b50d-205e1d79f224 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:59.124215 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6eb3d50d-45ea-4d15-b50d-205e1d79f224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:59.183128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:59.189879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385860e7-356b-4600-b8dc-b3db2940037a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:59.190927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 27/51] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:11:59.225911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ff44-03d8-42bd-9122-62f33ccb7b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:59.226514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ff44-03d8-42bd-9122-62f33ccb7b6b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.226794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ff44-03d8-42bd-9122-62f33ccb7b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.227065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ff44-03d8-42bd-9122-62f33ccb7b6b None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.227751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1ff44-03d8-42bd-9122-62f33ccb7b6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:59.228062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1ff44-03d8-42bd-9122-62f33ccb7b6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:59.229440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1ff44-03d8-42bd-9122-62f33ccb7b6b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:59.237629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 25/52] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:59.244262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3d50d-45ea-4d15-b50d-205e1d79f224 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:59.359543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3d50d-45ea-4d15-b50d-205e1d79f224 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:59.392311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2248a-e1ea-4a5e-8b50-736b3e77704c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:59.392938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2248a-e1ea-4a5e-8b50-736b3e77704c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.393220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2248a-e1ea-4a5e-8b50-736b3e77704c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.393557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2248a-e1ea-4a5e-8b50-736b3e77704c None admin] PATH_INFO: `/v3/users/9d4d38ef7ecd40a380aba27a2b564d42` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.394814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2248a-e1ea-4a5e-8b50-736b3e77704c None admin] RBAC: Authorizing `identity:get_user(user_id=9d4d38ef7ecd40a380aba27a2b564d42)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:59.396784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2248a-e1ea-4a5e-8b50-736b3e77704c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:59.398544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 28/53] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:11:59 2022] GET /identity/v3/users/9d4d38ef7ecd40a380aba27a2b564d42 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:59.404231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ff44-03d8-42bd-9122-62f33ccb7b6b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:59.418168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:59.466498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2248a-e1ea-4a5e-8b50-736b3e77704c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:59.506673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ff44-03d8-42bd-9122-62f33ccb7b6b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:59.529413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:59.541088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3ac9e-1a49-413e-99a9-6decfbe5fb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:59.550101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3ac9e-1a49-413e-99a9-6decfbe5fb9d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.550101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3ac9e-1a49-413e-99a9-6decfbe5fb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.550101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3ac9e-1a49-413e-99a9-6decfbe5fb9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.550101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3ac9e-1a49-413e-99a9-6decfbe5fb9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:59.550101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3ac9e-1a49-413e-99a9-6decfbe5fb9d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:59.550101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 26/54] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:59.552126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3ac9e-1a49-413e-99a9-6decfbe5fb9d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:59.569362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2248a-e1ea-4a5e-8b50-736b3e77704c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:59.605308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7d835-99b6-4e0e-9bba-53714ed38f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:59.606092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7d835-99b6-4e0e-9bba-53714ed38f12 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.606416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7d835-99b6-4e0e-9bba-53714ed38f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.606686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7d835-99b6-4e0e-9bba-53714ed38f12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.607782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7d835-99b6-4e0e-9bba-53714ed38f12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:59.609266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7d835-99b6-4e0e-9bba-53714ed38f12 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:59.610849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 29/55] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:59.616592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7d835-99b6-4e0e-9bba-53714ed38f12 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:59.633917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:59.657228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3ac9e-1a49-413e-99a9-6decfbe5fb9d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:59.694732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722aeb4-4ec6-4cab-ad73-5540c4114f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:59.695349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722aeb4-4ec6-4cab-ad73-5540c4114f64 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.695628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722aeb4-4ec6-4cab-ad73-5540c4114f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.695896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722aeb4-4ec6-4cab-ad73-5540c4114f64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.696748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722aeb4-4ec6-4cab-ad73-5540c4114f64 None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.domain.id=default, user.id=9d4d38ef7ecd40a380aba27a2b564d42) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:59.697053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722aeb4-4ec6-4cab-ad73-5540c4114f64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:59.698702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722aeb4-4ec6-4cab-ad73-5540c4114f64 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:59.705816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 27/56] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Sep 1 05:11:59 2022] GET /identity/v3/role_assignments?user.id=9d4d38ef7ecd40a380aba27a2b564d42&scope.domain.id=default&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:59.828866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7d835-99b6-4e0e-9bba-53714ed38f12 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:59.860598 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:59.867279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97853609-4697-44e6-9f9e-f2ed24946c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:59.871376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97853609-4697-44e6-9f9e-f2ed24946c56 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.871376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97853609-4697-44e6-9f9e-f2ed24946c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.871376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97853609-4697-44e6-9f9e-f2ed24946c56 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.871376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97853609-4697-44e6-9f9e-f2ed24946c56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:59.871376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97853609-4697-44e6-9f9e-f2ed24946c56 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:59.945208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 30/57] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Sep 1 05:11:59 2022] POST /identity/v3/projects => generated 298 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:12:01.857718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c5f84-fb94-4b6d-b13a-31fbae615cca None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:01.858224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c5f84-fb94-4b6d-b13a-31fbae615cca None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:01.858309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c5f84-fb94-4b6d-b13a-31fbae615cca None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:01.859630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 28/58] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:01.865250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364341f4-469f-42ff-9b60-4dfd116e554b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:01.865648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364341f4-469f-42ff-9b60-4dfd116e554b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:01.865861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364341f4-469f-42ff-9b60-4dfd116e554b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:01.883446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-364341f4-469f-42ff-9b60-4dfd116e554b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:01.884618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-364341f4-469f-42ff-9b60-4dfd116e554b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:01.948625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 31/59] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:02.217351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c5f84-fb94-4b6d-b13a-31fbae615cca None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:02.319148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c5f84-fb94-4b6d-b13a-31fbae615cca None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:02.351104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec44570-35ac-4b95-a86c-387a220f5ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:02.351718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec44570-35ac-4b95-a86c-387a220f5ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:02.351998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec44570-35ac-4b95-a86c-387a220f5ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:02.352270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec44570-35ac-4b95-a86c-387a220f5ba5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:02.353365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec44570-35ac-4b95-a86c-387a220f5ba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:02.354832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec44570-35ac-4b95-a86c-387a220f5ba5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:02.356358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 29/60] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:02.361821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364341f4-469f-42ff-9b60-4dfd116e554b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:02.457791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364341f4-469f-42ff-9b60-4dfd116e554b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:02.470506 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:02.491347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644a362-dda1-4013-b906-bc9cb7ea0b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:02.491939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644a362-dda1-4013-b906-bc9cb7ea0b20 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:02.492219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644a362-dda1-4013-b906-bc9cb7ea0b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:02.492493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644a362-dda1-4013-b906-bc9cb7ea0b20 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:02.493142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644a362-dda1-4013-b906-bc9cb7ea0b20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:02.494608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644a362-dda1-4013-b906-bc9cb7ea0b20 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:02.588512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 32/61] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:02 2022] POST /identity/v3/users => generated 275 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:05.677977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b22d9e-39b0-4a7a-963f-d5bf90e6e0b5 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.678488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b22d9e-39b0-4a7a-963f-d5bf90e6e0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.678594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b22d9e-39b0-4a7a-963f-d5bf90e6e0b5 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.680155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 30/62] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:05.685439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb274390-cbcb-4acd-8a88-b7146f8c0ded None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.685847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb274390-cbcb-4acd-8a88-b7146f8c0ded None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.686022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb274390-cbcb-4acd-8a88-b7146f8c0ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.714348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cb274390-cbcb-4acd-8a88-b7146f8c0ded None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:05.715572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cb274390-cbcb-4acd-8a88-b7146f8c0ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:05.824768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 33/63] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:05.842456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dbe71-1623-47cd-90bc-9ecc84f06733 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.842767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dbe71-1623-47cd-90bc-9ecc84f06733 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.843020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dbe71-1623-47cd-90bc-9ecc84f06733 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.844503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 31/64] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:05.850460 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4115a04-4944-48e7-9992-f18e7cbddf00 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.850787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4115a04-4944-48e7-9992-f18e7cbddf00 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.851031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4115a04-4944-48e7-9992-f18e7cbddf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.878864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b4115a04-4944-48e7-9992-f18e7cbddf00 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:05.880591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b4115a04-4944-48e7-9992-f18e7cbddf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:05.970833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 34/65] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:05.971149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818f407-0ecf-443b-8896-38f2c300a89a None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.971443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818f407-0ecf-443b-8896-38f2c300a89a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.971972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818f407-0ecf-443b-8896-38f2c300a89a None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.973853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 32/66] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:05.979279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25ede3-1a51-42a8-899f-d6ccdb1d41a8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.979567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25ede3-1a51-42a8-899f-d6ccdb1d41a8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.979847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25ede3-1a51-42a8-899f-d6ccdb1d41a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.006586 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5c25ede3-1a51-42a8-899f-d6ccdb1d41a8 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:06.009037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f099ac8-e9af-4a0c-8769-97f16258eec7 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.009666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f099ac8-e9af-4a0c-8769-97f16258eec7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.010162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f099ac8-e9af-4a0c-8769-97f16258eec7 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.011701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 33/67] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:06.012490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5c25ede3-1a51-42a8-899f-d6ccdb1d41a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:06.016722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0d411-8c96-4bd2-9f9b-dfa26d9afe71 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.017221 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0d411-8c96-4bd2-9f9b-dfa26d9afe71 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.017726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0d411-8c96-4bd2-9f9b-dfa26d9afe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.049045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b9b0d411-8c96-4bd2-9f9b-dfa26d9afe71 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:06.051501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b9b0d411-8c96-4bd2-9f9b-dfa26d9afe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:06.065791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.069852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.152405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 35/68] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:06.153525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25ede3-1a51-42a8-899f-d6ccdb1d41a8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.178157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 34/69] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:06.262731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25ede3-1a51-42a8-899f-d6ccdb1d41a8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.298954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e29cbf-9cdb-4b7e-a235-cb0fb04b8a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.299880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e29cbf-9cdb-4b7e-a235-cb0fb04b8a6c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.300341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e29cbf-9cdb-4b7e-a235-cb0fb04b8a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.300790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e29cbf-9cdb-4b7e-a235-cb0fb04b8a6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.312037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e29cbf-9cdb-4b7e-a235-cb0fb04b8a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-b3e29cbf-9cdb-4b7e-a235-cb0fb04b8a6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:06.317223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:06.318651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:06.319436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:06.321320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 36/70] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:06.327418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0d411-8c96-4bd2-9f9b-dfa26d9afe71 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.330590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e29cbf-9cdb-4b7e-a235-cb0fb04b8a6c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.351388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.437015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e29cbf-9cdb-4b7e-a235-cb0fb04b8a6c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.475935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b585c-12a4-45e6-8754-04969c7b9a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.476830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b585c-12a4-45e6-8754-04969c7b9a63 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.477353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b585c-12a4-45e6-8754-04969c7b9a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.477934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b585c-12a4-45e6-8754-04969c7b9a63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.480864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0d411-8c96-4bd2-9f9b-dfa26d9afe71 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.490542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b585c-12a4-45e6-8754-04969c7b9a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-568b585c-12a4-45e6-8754-04969c7b9a63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:06.495789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:06.496857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:06.497910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:06.499931 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 37/71] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:06.501427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a19929-a799-4ea9-ab52-42f322da8b1d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.501923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a19929-a799-4ea9-ab52-42f322da8b1d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.502438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a19929-a799-4ea9-ab52-42f322da8b1d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.504533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 38/72] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:06.506113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a19929-a799-4ea9-ab52-42f322da8b1d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.523052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15270b7-6061-4ac1-9773-5adbe7e2369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.523599 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15270b7-6061-4ac1-9773-5adbe7e2369f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.524125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15270b7-6061-4ac1-9773-5adbe7e2369f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.524125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15270b7-6061-4ac1-9773-5adbe7e2369f None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.524798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15270b7-6061-4ac1-9773-5adbe7e2369f None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:06.525099 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15270b7-6061-4ac1-9773-5adbe7e2369f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:06.526518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15270b7-6061-4ac1-9773-5adbe7e2369f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:06.535232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 35/73] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:06.536917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44d01c-b184-4367-8787-f19023e2a67b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.537237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44d01c-b184-4367-8787-f19023e2a67b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.537483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44d01c-b184-4367-8787-f19023e2a67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.543625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.556405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8f44d01c-b184-4367-8787-f19023e2a67b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:06.558257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8f44d01c-b184-4367-8787-f19023e2a67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:06.596340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.620141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a19929-a799-4ea9-ab52-42f322da8b1d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.626914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 36/74] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:06.637606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44d01c-b184-4367-8787-f19023e2a67b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.667301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130ddbc-4dd8-4e0d-82f4-e9e33a6c3d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.667921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130ddbc-4dd8-4e0d-82f4-e9e33a6c3d50 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.668206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130ddbc-4dd8-4e0d-82f4-e9e33a6c3d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.668483 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130ddbc-4dd8-4e0d-82f4-e9e33a6c3d50 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.669152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130ddbc-4dd8-4e0d-82f4-e9e33a6c3d50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:06.669478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130ddbc-4dd8-4e0d-82f4-e9e33a6c3d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:06.670879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130ddbc-4dd8-4e0d-82f4-e9e33a6c3d50 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:06.682962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 39/75] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:06.683599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130ddbc-4dd8-4e0d-82f4-e9e33a6c3d50 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.739691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44d01c-b184-4367-8787-f19023e2a67b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.772883 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.778180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f2374-c680-43b5-9c70-c6825a363291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.778758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f2374-c680-43b5-9c70-c6825a363291 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.779036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f2374-c680-43b5-9c70-c6825a363291 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.779370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f2374-c680-43b5-9c70-c6825a363291 None admin] PATH_INFO: `/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.793515 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f2374-c680-43b5-9c70-c6825a363291 None admin] RBAC: Authorizing `identity:get_user(user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:06.794966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f2374-c680-43b5-9c70-c6825a363291 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:06.796594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 37/76] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:06.800561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130ddbc-4dd8-4e0d-82f4-e9e33a6c3d50 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.802859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f2374-c680-43b5-9c70-c6825a363291 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.841716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c11fc-8ab1-4465-8596-caac6b0c50dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.842413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c11fc-8ab1-4465-8596-caac6b0c50dd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.842704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c11fc-8ab1-4465-8596-caac6b0c50dd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.843028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c11fc-8ab1-4465-8596-caac6b0c50dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.855192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.856444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c11fc-8ab1-4465-8596-caac6b0c50dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-948c11fc-8ab1-4465-8596-caac6b0c50dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:06.861689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:06.862591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:06.863424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 40/77] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:06.864111 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c11fc-8ab1-4465-8596-caac6b0c50dd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.918168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f2374-c680-43b5-9c70-c6825a363291 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.953169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e59a3-effd-4d3c-aafe-a1a0ceababc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.953829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e59a3-effd-4d3c-aafe-a1a0ceababc4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.954110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e59a3-effd-4d3c-aafe-a1a0ceababc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.954385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e59a3-effd-4d3c-aafe-a1a0ceababc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.964933 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e59a3-effd-4d3c-aafe-a1a0ceababc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:06.968314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c11fc-8ab1-4465-8596-caac6b0c50dd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-074e59a3-effd-4d3c-aafe-a1a0ceababc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:06.969737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:06.970756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:06.971506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 38/78] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:06.972322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e59a3-effd-4d3c-aafe-a1a0ceababc4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.007796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083cba5-aa99-46d3-adc0-8c04026911c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.008437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083cba5-aa99-46d3-adc0-8c04026911c6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.008719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083cba5-aa99-46d3-adc0-8c04026911c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.008999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083cba5-aa99-46d3-adc0-8c04026911c6 None admin] PATH_INFO: `/v3/users/9d4d38ef7ecd40a380aba27a2b564d42` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.010171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083cba5-aa99-46d3-adc0-8c04026911c6 None admin] RBAC: Authorizing `identity:get_user(user_id=9d4d38ef7ecd40a380aba27a2b564d42)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.011643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083cba5-aa99-46d3-adc0-8c04026911c6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.013241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 41/79] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/users/9d4d38ef7ecd40a380aba27a2b564d42 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:07.014040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083cba5-aa99-46d3-adc0-8c04026911c6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.022302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.072208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e59a3-effd-4d3c-aafe-a1a0ceababc4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.079620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.106324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd9858-fb09-4cfe-aff1-62aee5a1b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.106935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd9858-fb09-4cfe-aff1-62aee5a1b09b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.107258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd9858-fb09-4cfe-aff1-62aee5a1b09b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.107533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd9858-fb09-4cfe-aff1-62aee5a1b09b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.112662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083cba5-aa99-46d3-adc0-8c04026911c6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.147542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9c8e1-9647-472a-b9e4-104608a97324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.148145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9c8e1-9647-472a-b9e4-104608a97324 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.148429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9c8e1-9647-472a-b9e4-104608a97324 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.148702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9c8e1-9647-472a-b9e4-104608a97324 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.149363 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9c8e1-9647-472a-b9e4-104608a97324 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:07.149701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9c8e1-9647-472a-b9e4-104608a97324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.151071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9c8e1-9647-472a-b9e4-104608a97324 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.159249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 42/80] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:07.160036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9c8e1-9647-472a-b9e4-104608a97324 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.184110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd9858-fb09-4cfe-aff1-62aee5a1b09b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.185668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd9858-fb09-4cfe-aff1-62aee5a1b09b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.235431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.260574 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9c8e1-9647-472a-b9e4-104608a97324 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.288082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 39/81] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:07.288699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.290347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd9858-fb09-4cfe-aff1-62aee5a1b09b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.294703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941679a4-d920-4d19-abba-bb61dc584bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.295352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941679a4-d920-4d19-abba-bb61dc584bfb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.295633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941679a4-d920-4d19-abba-bb61dc584bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.295907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941679a4-d920-4d19-abba-bb61dc584bfb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.297017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941679a4-d920-4d19-abba-bb61dc584bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.298575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941679a4-d920-4d19-abba-bb61dc584bfb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.300152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 43/82] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:07.300909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941679a4-d920-4d19-abba-bb61dc584bfb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.392258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd9858-fb09-4cfe-aff1-62aee5a1b09b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.409027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941679a4-d920-4d19-abba-bb61dc584bfb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.425911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45816b5f-15f2-48f9-bb37-a0d93f0b5f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.426522 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45816b5f-15f2-48f9-bb37-a0d93f0b5f0d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.426804 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45816b5f-15f2-48f9-bb37-a0d93f0b5f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.427636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45816b5f-15f2-48f9-bb37-a0d93f0b5f0d None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.428395 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45816b5f-15f2-48f9-bb37-a0d93f0b5f0d None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:07.428704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45816b5f-15f2-48f9-bb37-a0d93f0b5f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.430137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45816b5f-15f2-48f9-bb37-a0d93f0b5f0d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.438061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 40/83] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:07.438850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45816b5f-15f2-48f9-bb37-a0d93f0b5f0d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.448940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0d18e-0ec2-4f8e-97fe-857a9e99ad8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.449591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0d18e-0ec2-4f8e-97fe-857a9e99ad8d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.449876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0d18e-0ec2-4f8e-97fe-857a9e99ad8d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.450764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0d18e-0ec2-4f8e-97fe-857a9e99ad8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.454235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.515984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.519319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0d18e-0ec2-4f8e-97fe-857a9e99ad8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.520949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0d18e-0ec2-4f8e-97fe-857a9e99ad8d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.538391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45816b5f-15f2-48f9-bb37-a0d93f0b5f0d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.573003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf5879-eaf6-4193-9747-300dab329e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.573665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf5879-eaf6-4193-9747-300dab329e1e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.573947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf5879-eaf6-4193-9747-300dab329e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.574220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf5879-eaf6-4193-9747-300dab329e1e None admin] PATH_INFO: `/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.575615 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf5879-eaf6-4193-9747-300dab329e1e None admin] RBAC: Authorizing `identity:get_user(user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.577073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf5879-eaf6-4193-9747-300dab329e1e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.578876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 41/84] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:07.579446 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf5879-eaf6-4193-9747-300dab329e1e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.623104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 44/85] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:07.623927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0d18e-0ec2-4f8e-97fe-857a9e99ad8d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.660543 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.679566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf5879-eaf6-4193-9747-300dab329e1e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.717789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86c917-8a12-496d-aeec-b8dc30bd2528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.718423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86c917-8a12-496d-aeec-b8dc30bd2528 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.718703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86c917-8a12-496d-aeec-b8dc30bd2528 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.718976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86c917-8a12-496d-aeec-b8dc30bd2528 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.720208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86c917-8a12-496d-aeec-b8dc30bd2528 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.720384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.721583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86c917-8a12-496d-aeec-b8dc30bd2528 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.723219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 42/86] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:07.724395 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86c917-8a12-496d-aeec-b8dc30bd2528 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.724771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0d18e-0ec2-4f8e-97fe-857a9e99ad8d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.759771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f14422-071e-419d-bf11-77caaf3d6483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.760386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f14422-071e-419d-bf11-77caaf3d6483 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.760657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f14422-071e-419d-bf11-77caaf3d6483 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.760926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f14422-071e-419d-bf11-77caaf3d6483 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.761599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f14422-071e-419d-bf11-77caaf3d6483 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.763046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f14422-071e-419d-bf11-77caaf3d6483 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.824597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86c917-8a12-496d-aeec-b8dc30bd2528 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.845254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 45/87] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:07 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:07.846133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f14422-071e-419d-bf11-77caaf3d6483 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.860517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19cb1d-02ff-40af-888e-952a5432652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.861185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19cb1d-02ff-40af-888e-952a5432652b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.861489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19cb1d-02ff-40af-888e-952a5432652b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.861814 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19cb1d-02ff-40af-888e-952a5432652b None admin] PATH_INFO: `/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.865594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19cb1d-02ff-40af-888e-952a5432652b None admin] RBAC: Authorizing `identity:get_user(user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.865594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19cb1d-02ff-40af-888e-952a5432652b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.866775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 43/88] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:07.867351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19cb1d-02ff-40af-888e-952a5432652b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.905812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.986089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f14422-071e-419d-bf11-77caaf3d6483 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.999542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19cb1d-02ff-40af-888e-952a5432652b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.027852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f5247-b8fa-431b-a70a-aafe2b8eab3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.028476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f5247-b8fa-431b-a70a-aafe2b8eab3d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.028758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f5247-b8fa-431b-a70a-aafe2b8eab3d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.029029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f5247-b8fa-431b-a70a-aafe2b8eab3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.045181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a6379-b6a2-4512-b0e5-bcf6c4675364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.045891 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a6379-b6a2-4512-b0e5-bcf6c4675364 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.046175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a6379-b6a2-4512-b0e5-bcf6c4675364 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.046449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a6379-b6a2-4512-b0e5-bcf6c4675364 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.048549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a6379-b6a2-4512-b0e5-bcf6c4675364 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.050043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a6379-b6a2-4512-b0e5-bcf6c4675364 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.053103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 44/89] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:08.053103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a6379-b6a2-4512-b0e5-bcf6c4675364 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.063526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.123487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f5247-b8fa-431b-a70a-aafe2b8eab3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.125792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f5247-b8fa-431b-a70a-aafe2b8eab3d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.170521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.195151 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a6379-b6a2-4512-b0e5-bcf6c4675364 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.247007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e73722-38fb-4359-94c5-132a65c7001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.247554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73722-38fb-4359-94c5-132a65c7001b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.248086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73722-38fb-4359-94c5-132a65c7001b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.248379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e73722-38fb-4359-94c5-132a65c7001b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.249370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e73722-38fb-4359-94c5-132a65c7001b None admin] RBAC: Adding query filter params (role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, scope.project.id=812094fb94404914876df4528f073add, user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:08.249705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e73722-38fb-4359-94c5-132a65c7001b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.251852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e73722-38fb-4359-94c5-132a65c7001b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.259962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 45/90] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/role_assignments?user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae&scope.project.id=812094fb94404914876df4528f073add&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:08.260788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e73722-38fb-4359-94c5-132a65c7001b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.261289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 46/91] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:08.266473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f5247-b8fa-431b-a70a-aafe2b8eab3d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.377521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.433286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f5247-b8fa-431b-a70a-aafe2b8eab3d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.493416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d231d-30c7-4dcb-853d-f78c22eb70f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.494074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d231d-30c7-4dcb-853d-f78c22eb70f1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.494357 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d231d-30c7-4dcb-853d-f78c22eb70f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.494637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d231d-30c7-4dcb-853d-f78c22eb70f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.495385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d231d-30c7-4dcb-853d-f78c22eb70f1 None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=812094fb94404914876df4528f073add, user.id=9d4d38ef7ecd40a380aba27a2b564d42) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:08.495695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d231d-30c7-4dcb-853d-f78c22eb70f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.497227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d231d-30c7-4dcb-853d-f78c22eb70f1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.500024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e73722-38fb-4359-94c5-132a65c7001b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.514381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 47/92] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/role_assignments?user.id=9d4d38ef7ecd40a380aba27a2b564d42&scope.project.id=812094fb94404914876df4528f073add&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:08.515307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d231d-30c7-4dcb-853d-f78c22eb70f1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.598196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001bf2f-7679-46c6-918a-026338fafcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.599273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001bf2f-7679-46c6-918a-026338fafcdb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.599708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001bf2f-7679-46c6-918a-026338fafcdb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.600110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001bf2f-7679-46c6-918a-026338fafcdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.648360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d231d-30c7-4dcb-853d-f78c22eb70f1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.650545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.681855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.727550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12862437-1e7c-4962-a3dd-6e23de8a0fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.728995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12862437-1e7c-4962-a3dd-6e23de8a0fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.729479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12862437-1e7c-4962-a3dd-6e23de8a0fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.729920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12862437-1e7c-4962-a3dd-6e23de8a0fe0 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.731540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12862437-1e7c-4962-a3dd-6e23de8a0fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.733792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12862437-1e7c-4962-a3dd-6e23de8a0fe0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.735807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 48/93] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:08.742961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12862437-1e7c-4962-a3dd-6e23de8a0fe0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.778397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001bf2f-7679-46c6-918a-026338fafcdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.780104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001bf2f-7679-46c6-918a-026338fafcdb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.932747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12862437-1e7c-4962-a3dd-6e23de8a0fe0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.961285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 46/94] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:08.966822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001bf2f-7679-46c6-918a-026338fafcdb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:09.003081 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa27286-2b65-47ff-aa60-28ffb60af90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:09.004083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa27286-2b65-47ff-aa60-28ffb60af90b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.004566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa27286-2b65-47ff-aa60-28ffb60af90b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.005085 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa27286-2b65-47ff-aa60-28ffb60af90b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.006178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa27286-2b65-47ff-aa60-28ffb60af90b None admin] RBAC: Adding query filter params (role.id=acfe3eeb91964f62abb1d8a84c36dccc, scope.project.id=812094fb94404914876df4528f073add, user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:09.006692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa27286-2b65-47ff-aa60-28ffb60af90b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:09.008425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa27286-2b65-47ff-aa60-28ffb60af90b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:09.019493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 49/95] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/role_assignments?user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae&scope.project.id=812094fb94404914876df4528f073add&role.id=acfe3eeb91964f62abb1d8a84c36dccc => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:09.030118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:09.097562 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001bf2f-7679-46c6-918a-026338fafcdb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:09.137928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ff54a-166c-4689-8da5-72f76071437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:09.138579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ff54a-166c-4689-8da5-72f76071437c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.138857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ff54a-166c-4689-8da5-72f76071437c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.139125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ff54a-166c-4689-8da5-72f76071437c None admin] PATH_INFO: `/v3/projects/53e1686defa64da4a51a16f90ce83c39` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.140419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ff54a-166c-4689-8da5-72f76071437c None admin] RBAC: Authorizing `identity:get_project(project_id=53e1686defa64da4a51a16f90ce83c39)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:09.141948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ff54a-166c-4689-8da5-72f76071437c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:09.143925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 47/96] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/projects/53e1686defa64da4a51a16f90ce83c39 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:09.151080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa27286-2b65-47ff-aa60-28ffb60af90b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:09.207617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:09.284731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa27286-2b65-47ff-aa60-28ffb60af90b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:09.331261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec339d-b886-48ec-96db-7c8c062a52ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:09.331878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec339d-b886-48ec-96db-7c8c062a52ea None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.332159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec339d-b886-48ec-96db-7c8c062a52ea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.332434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec339d-b886-48ec-96db-7c8c062a52ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.333803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec339d-b886-48ec-96db-7c8c062a52ea None admin] RBAC: Adding query filter params (role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, scope.project.id=53e1686defa64da4a51a16f90ce83c39, user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:09.334356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec339d-b886-48ec-96db-7c8c062a52ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:09.335863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec339d-b886-48ec-96db-7c8c062a52ea None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:09.343431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 50/97] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:09 2022] GET /identity/v3/role_assignments?user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae&scope.project.id=53e1686defa64da4a51a16f90ce83c39&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:10.335425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df848f0-f3f6-414d-9ecd-67c9c590e95a None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:10.335972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df848f0-f3f6-414d-9ecd-67c9c590e95a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:10.335972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df848f0-f3f6-414d-9ecd-67c9c590e95a None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:10.337311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 48/98] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:10.342680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0cc8d-1810-4d0a-88a0-86bdca2f7ee2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:10.343079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0cc8d-1810-4d0a-88a0-86bdca2f7ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:10.343258 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0cc8d-1810-4d0a-88a0-86bdca2f7ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:10.371492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c6f0cc8d-1810-4d0a-88a0-86bdca2f7ee2 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:10.372719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c6f0cc8d-1810-4d0a-88a0-86bdca2f7ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:10.473570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 51/99] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:10.556238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df525abc-5e7e-41c3-94b8-b6656a29d08b None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:10.556574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df525abc-5e7e-41c3-94b8-b6656a29d08b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:10.556801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df525abc-5e7e-41c3-94b8-b6656a29d08b None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:10.558229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 49/100] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:10.563504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c8a40-3582-4872-91dd-b13fcbee8ec4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:10.563804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c8a40-3582-4872-91dd-b13fcbee8ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:10.564090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c8a40-3582-4872-91dd-b13fcbee8ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:10.582762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ea2c8a40-3582-4872-91dd-b13fcbee8ec4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:10.584477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ea2c8a40-3582-4872-91dd-b13fcbee8ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:10.626484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:10.654593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 52/101] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:10.959417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df525abc-5e7e-41c3-94b8-b6656a29d08b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:11.002227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c8a40-3582-4872-91dd-b13fcbee8ec4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:11.113588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df525abc-5e7e-41c3-94b8-b6656a29d08b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:11.139640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:11.172031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770ee2b-9b81-4b0a-ad70-093cc80fc8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:11.173001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770ee2b-9b81-4b0a-ad70-093cc80fc8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.173540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770ee2b-9b81-4b0a-ad70-093cc80fc8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.174005 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770ee2b-9b81-4b0a-ad70-093cc80fc8d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.175275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770ee2b-9b81-4b0a-ad70-093cc80fc8d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:11.177976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770ee2b-9b81-4b0a-ad70-093cc80fc8d9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:11.180199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 50/102] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:11.186798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770ee2b-9b81-4b0a-ad70-093cc80fc8d9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:11.263710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c8a40-3582-4872-91dd-b13fcbee8ec4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:11.366226 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f8ce6-a1a7-4fe3-8acf-08aabd55f219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:11.366967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f8ce6-a1a7-4fe3-8acf-08aabd55f219 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.367150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f8ce6-a1a7-4fe3-8acf-08aabd55f219 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.367427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f8ce6-a1a7-4fe3-8acf-08aabd55f219 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.384553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f8ce6-a1a7-4fe3-8acf-08aabd55f219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-695f8ce6-a1a7-4fe3-8acf-08aabd55f219 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:11.389367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:11.390319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:11.391067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 53/103] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:11.392440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a570f-9813-4d61-afeb-f39a8f8cb049 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.392440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a570f-9813-4d61-afeb-f39a8f8cb049 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.392671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a570f-9813-4d61-afeb-f39a8f8cb049 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.399484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770ee2b-9b81-4b0a-ad70-093cc80fc8d9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:11.402485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 54/104] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:11 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:11.410631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a570f-9813-4d61-afeb-f39a8f8cb049 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:11.455890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe63a64-cebb-4926-bed8-5c79deb889e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:11.456507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe63a64-cebb-4926-bed8-5c79deb889e7 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.456783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe63a64-cebb-4926-bed8-5c79deb889e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.457051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe63a64-cebb-4926-bed8-5c79deb889e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.457734 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe63a64-cebb-4926-bed8-5c79deb889e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:11.459237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe63a64-cebb-4926-bed8-5c79deb889e7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:11.479526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:11.553622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a570f-9813-4d61-afeb-f39a8f8cb049 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:11.573385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 51/105] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:12:11 2022] POST /identity/v3/projects => generated 302 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:11.574739 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaecc4-53a9-41fa-b529-6fcec1fd3272 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.575335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaecc4-53a9-41fa-b529-6fcec1fd3272 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.575613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaecc4-53a9-41fa-b529-6fcec1fd3272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.611243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c813d33-9be0-4e72-bad2-e8dc7c36cfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:11.611947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c813d33-9be0-4e72-bad2-e8dc7c36cfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.612365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c813d33-9be0-4e72-bad2-e8dc7c36cfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.612644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c813d33-9be0-4e72-bad2-e8dc7c36cfb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.613556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c813d33-9be0-4e72-bad2-e8dc7c36cfb3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:11.613870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c813d33-9be0-4e72-bad2-e8dc7c36cfb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:11.615686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c813d33-9be0-4e72-bad2-e8dc7c36cfb3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:11.618662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-dfeaecc4-53a9-41fa-b529-6fcec1fd3272 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:11.619935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dfeaecc4-53a9-41fa-b529-6fcec1fd3272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:11.626720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 55/106] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:11.628365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa3ab5-a12e-456b-a90f-a835d0b208df None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.632843 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa3ab5-a12e-456b-a90f-a835d0b208df None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.633579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa3ab5-a12e-456b-a90f-a835d0b208df None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.635583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 56/107] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:11.638209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa3ab5-a12e-456b-a90f-a835d0b208df None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:11.684790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:11.764604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 52/108] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:11.765927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c65b19-945a-4695-a05e-390729c2f871 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.767388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c65b19-945a-4695-a05e-390729c2f871 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.767664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c65b19-945a-4695-a05e-390729c2f871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.788312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-57c65b19-945a-4695-a05e-390729c2f871 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:11.789681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-57c65b19-945a-4695-a05e-390729c2f871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:11.809095 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:11.818818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa3ab5-a12e-456b-a90f-a835d0b208df None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:11.862070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7905e4-94eb-4a28-9034-e2732a4b3a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:11.862891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7905e4-94eb-4a28-9034-e2732a4b3a29 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.863276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7905e4-94eb-4a28-9034-e2732a4b3a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.863659 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7905e4-94eb-4a28-9034-e2732a4b3a29 None admin] PATH_INFO: `/v3/users/a2d376e28f7f47888a22de4d938c2cfe` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.880724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 53/109] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:11.896484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7905e4-94eb-4a28-9034-e2732a4b3a29 None admin] RBAC: Authorizing `identity:get_user(user_id=a2d376e28f7f47888a22de4d938c2cfe)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:11.898128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7905e4-94eb-4a28-9034-e2732a4b3a29 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:11.899898 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 57/110] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:11 2022] GET /identity/v3/users/a2d376e28f7f47888a22de4d938c2cfe => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:11.905731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c65b19-945a-4695-a05e-390729c2f871 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.032646 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.039360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c65b19-945a-4695-a05e-390729c2f871 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.092589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fae18c-71cc-4443-9948-83a7a5095b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.093220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fae18c-71cc-4443-9948-83a7a5095b89 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.093519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fae18c-71cc-4443-9948-83a7a5095b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.093798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fae18c-71cc-4443-9948-83a7a5095b89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.118877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7905e4-94eb-4a28-9034-e2732a4b3a29 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.197826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fae18c-71cc-4443-9948-83a7a5095b89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.199721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fae18c-71cc-4443-9948-83a7a5095b89 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.300539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7905e4-94eb-4a28-9034-e2732a4b3a29 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.311428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.318377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 54/111] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:12.324607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ada78-761b-475b-8b0c-c354c08bb74f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.325569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ada78-761b-475b-8b0c-c354c08bb74f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.326078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ada78-761b-475b-8b0c-c354c08bb74f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.327611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 55/112] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:12.328971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b0ca-8244-415e-aa49-23d831f446a3 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.329418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b0ca-8244-415e-aa49-23d831f446a3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.329880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6b0ca-8244-415e-aa49-23d831f446a3 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.331376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 56/113] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:12.336702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6b0ca-8244-415e-aa49-23d831f446a3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.336819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848e06a-c0ed-4768-a4ca-184023ff8002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.337221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848e06a-c0ed-4768-a4ca-184023ff8002 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.337531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848e06a-c0ed-4768-a4ca-184023ff8002 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.337835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848e06a-c0ed-4768-a4ca-184023ff8002 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.350296 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848e06a-c0ed-4768-a4ca-184023ff8002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5848e06a-c0ed-4768-a4ca-184023ff8002 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:12.355929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:12.356777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:12.357568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:12.359519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 58/114] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:12.360810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f675d40-2b51-4ea6-975c-13216c7f9a1c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.361253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f675d40-2b51-4ea6-975c-13216c7f9a1c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.361710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f675d40-2b51-4ea6-975c-13216c7f9a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.380075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7f675d40-2b51-4ea6-975c-13216c7f9a1c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:12.381424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7f675d40-2b51-4ea6-975c-13216c7f9a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:12.391528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.452358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6b0ca-8244-415e-aa49-23d831f446a3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.454657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 59/115] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:12.456542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b16f4-7e7d-4599-af06-71a2f6143ee1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.456837 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b16f4-7e7d-4599-af06-71a2f6143ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.457118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b16f4-7e7d-4599-af06-71a2f6143ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.481426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ac6b16f4-7e7d-4599-af06-71a2f6143ee1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:12.483053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ac6b16f4-7e7d-4599-af06-71a2f6143ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:12.493904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552facc9-3f5d-4097-8cde-298dc3b5e3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.494976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552facc9-3f5d-4097-8cde-298dc3b5e3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.495318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552facc9-3f5d-4097-8cde-298dc3b5e3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.495653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552facc9-3f5d-4097-8cde-298dc3b5e3b0 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.497173 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552facc9-3f5d-4097-8cde-298dc3b5e3b0 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.499049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552facc9-3f5d-4097-8cde-298dc3b5e3b0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.501295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 57/116] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:12.506332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552facc9-3f5d-4097-8cde-298dc3b5e3b0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.564651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 60/117] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:12.565956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b16f4-7e7d-4599-af06-71a2f6143ee1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.608484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.678514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552facc9-3f5d-4097-8cde-298dc3b5e3b0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.700722 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.741131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b16f4-7e7d-4599-af06-71a2f6143ee1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.741320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50178b4-749d-42c9-a898-ef6d6bde899c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.741911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50178b4-749d-42c9-a898-ef6d6bde899c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.742203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50178b4-749d-42c9-a898-ef6d6bde899c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.742484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50178b4-749d-42c9-a898-ef6d6bde899c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.756535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50178b4-749d-42c9-a898-ef6d6bde899c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d50178b4-749d-42c9-a898-ef6d6bde899c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:12.763437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:12.764285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:12.765070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 58/118] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:12.766565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50178b4-749d-42c9-a898-ef6d6bde899c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.780458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d59843-0d22-413d-8c58-ab6eac1ed983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.781319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d59843-0d22-413d-8c58-ab6eac1ed983 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.781792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d59843-0d22-413d-8c58-ab6eac1ed983 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.782227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d59843-0d22-413d-8c58-ab6eac1ed983 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.783112 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d59843-0d22-413d-8c58-ab6eac1ed983 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:12.783523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d59843-0d22-413d-8c58-ab6eac1ed983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.784993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d59843-0d22-413d-8c58-ab6eac1ed983 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.798394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 61/119] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:12.798394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d59843-0d22-413d-8c58-ab6eac1ed983 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.902706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50178b4-749d-42c9-a898-ef6d6bde899c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.910532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d59843-0d22-413d-8c58-ab6eac1ed983 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.930753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.939893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9749e8-ec22-4647-bdd1-02db4c3ded9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.940511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9749e8-ec22-4647-bdd1-02db4c3ded9c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.940793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9749e8-ec22-4647-bdd1-02db4c3ded9c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.941067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9749e8-ec22-4647-bdd1-02db4c3ded9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.941824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9749e8-ec22-4647-bdd1-02db4c3ded9c None admin] RBAC: Adding query filter params (role.id=d529d6d058424562a72162b0c257851f, scope.project.id=812094fb94404914876df4528f073add, user.id=a2d376e28f7f47888a22de4d938c2cfe) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:12.942132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9749e8-ec22-4647-bdd1-02db4c3ded9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.943635 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9749e8-ec22-4647-bdd1-02db4c3ded9c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.946550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62720c4b-760e-4d5d-b718-5fe70d24710b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.952429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62720c4b-760e-4d5d-b718-5fe70d24710b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.952429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62720c4b-760e-4d5d-b718-5fe70d24710b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.952429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62720c4b-760e-4d5d-b718-5fe70d24710b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.952672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 59/120] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/role_assignments?user.id=a2d376e28f7f47888a22de4d938c2cfe&scope.project.id=812094fb94404914876df4528f073add&role.id=d529d6d058424562a72162b0c257851f => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:12.952672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9749e8-ec22-4647-bdd1-02db4c3ded9c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.956149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.970960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62720c4b-760e-4d5d-b718-5fe70d24710b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-62720c4b-760e-4d5d-b718-5fe70d24710b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:12.976871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:12.977842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:12.978606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:12.981144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 62/121] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:12.982266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62720c4b-760e-4d5d-b718-5fe70d24710b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.086266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9749e8-ec22-4647-bdd1-02db4c3ded9c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.128141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d63b89-5199-414a-b05f-d89dee888f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.129113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d63b89-5199-414a-b05f-d89dee888f0a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.129639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d63b89-5199-414a-b05f-d89dee888f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.130645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d63b89-5199-414a-b05f-d89dee888f0a None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.131521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d63b89-5199-414a-b05f-d89dee888f0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:13.131701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62720c4b-760e-4d5d-b718-5fe70d24710b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.132193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d63b89-5199-414a-b05f-d89dee888f0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.133806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d63b89-5199-414a-b05f-d89dee888f0a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.144285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 60/122] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:13.145648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d63b89-5199-414a-b05f-d89dee888f0a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.181208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f73e66-27d9-4d6e-83a9-6e79c5c07cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.181208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f73e66-27d9-4d6e-83a9-6e79c5c07cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.181208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f73e66-27d9-4d6e-83a9-6e79c5c07cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.181208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f73e66-27d9-4d6e-83a9-6e79c5c07cb3 None admin] PATH_INFO: `/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.181604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f73e66-27d9-4d6e-83a9-6e79c5c07cb3 None admin] RBAC: Authorizing `identity:get_user(user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.181604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f73e66-27d9-4d6e-83a9-6e79c5c07cb3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.182793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 63/123] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:13.186481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f73e66-27d9-4d6e-83a9-6e79c5c07cb3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.206033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.266695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.273550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d63b89-5199-414a-b05f-d89dee888f0a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.306636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f73e66-27d9-4d6e-83a9-6e79c5c07cb3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.317760 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3d1d7-d391-476b-92bb-cfd74ee8b026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.318724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3d1d7-d391-476b-92bb-cfd74ee8b026 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.319019 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3d1d7-d391-476b-92bb-cfd74ee8b026 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.319298 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3d1d7-d391-476b-92bb-cfd74ee8b026 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.332731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3d1d7-d391-476b-92bb-cfd74ee8b026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-80b3d1d7-d391-476b-92bb-cfd74ee8b026 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:13.339389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:13.340277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:13.341060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 61/124] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:13.341729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3d1d7-d391-476b-92bb-cfd74ee8b026 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.350652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda62b92-5e48-45a5-a417-1e88758a43df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.351662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda62b92-5e48-45a5-a417-1e88758a43df None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.352141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda62b92-5e48-45a5-a417-1e88758a43df None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.352613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda62b92-5e48-45a5-a417-1e88758a43df None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.354054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda62b92-5e48-45a5-a417-1e88758a43df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:13.354578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda62b92-5e48-45a5-a417-1e88758a43df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.356643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda62b92-5e48-45a5-a417-1e88758a43df None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.369758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 64/125] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:13.370889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda62b92-5e48-45a5-a417-1e88758a43df None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.468184 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3d1d7-d391-476b-92bb-cfd74ee8b026 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.481897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.482823 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda62b92-5e48-45a5-a417-1e88758a43df None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.510104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.510593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e15f2-83a8-45f0-a940-5c31c816e358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.511960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e15f2-83a8-45f0-a940-5c31c816e358 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.512440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e15f2-83a8-45f0-a940-5c31c816e358 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.512949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e15f2-83a8-45f0-a940-5c31c816e358 None admin] PATH_INFO: `/v3/users/9d4d38ef7ecd40a380aba27a2b564d42` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.514323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e15f2-83a8-45f0-a940-5c31c816e358 None admin] RBAC: Authorizing `identity:get_user(user_id=9d4d38ef7ecd40a380aba27a2b564d42)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.516093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e15f2-83a8-45f0-a940-5c31c816e358 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.518434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 62/126] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/users/9d4d38ef7ecd40a380aba27a2b564d42 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:13.518910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed856e-a827-4f61-bb96-4b57b5ce6c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.519631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed856e-a827-4f61-bb96-4b57b5ce6c46 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.519915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed856e-a827-4f61-bb96-4b57b5ce6c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.520198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed856e-a827-4f61-bb96-4b57b5ce6c46 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.521243 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e15f2-83a8-45f0-a940-5c31c816e358 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.522362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed856e-a827-4f61-bb96-4b57b5ce6c46 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.524076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed856e-a827-4f61-bb96-4b57b5ce6c46 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.530646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 65/127] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:13.536032 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed856e-a827-4f61-bb96-4b57b5ce6c46 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.635318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e15f2-83a8-45f0-a940-5c31c816e358 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.636749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed856e-a827-4f61-bb96-4b57b5ce6c46 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.671745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285cff0-cd5a-4fdd-8f77-caa0ff14ef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.672453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285cff0-cd5a-4fdd-8f77-caa0ff14ef16 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.672740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285cff0-cd5a-4fdd-8f77-caa0ff14ef16 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.673084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285cff0-cd5a-4fdd-8f77-caa0ff14ef16 None admin] PATH_INFO: `/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.674429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285cff0-cd5a-4fdd-8f77-caa0ff14ef16 None admin] RBAC: Authorizing `identity:get_user(user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.675169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7185728-3adb-498c-8b74-6bbef5099555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.675904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285cff0-cd5a-4fdd-8f77-caa0ff14ef16 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.676161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7185728-3adb-498c-8b74-6bbef5099555 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.677386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7185728-3adb-498c-8b74-6bbef5099555 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.678046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7185728-3adb-498c-8b74-6bbef5099555 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.678451 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 66/128] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:13.679304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7185728-3adb-498c-8b74-6bbef5099555 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:13.679787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7185728-3adb-498c-8b74-6bbef5099555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.681404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7185728-3adb-498c-8b74-6bbef5099555 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.683498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285cff0-cd5a-4fdd-8f77-caa0ff14ef16 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.691315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 63/129] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:13.692478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7185728-3adb-498c-8b74-6bbef5099555 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.739980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.750629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.791113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285cff0-cd5a-4fdd-8f77-caa0ff14ef16 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.804219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7185728-3adb-498c-8b74-6bbef5099555 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.827030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94f8a3-2970-4ba9-9cff-dd5172aafba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.827726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f8a3-2970-4ba9-9cff-dd5172aafba9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.828022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f8a3-2970-4ba9-9cff-dd5172aafba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.828334 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f8a3-2970-4ba9-9cff-dd5172aafba9 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.829448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94f8a3-2970-4ba9-9cff-dd5172aafba9 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.831062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94f8a3-2970-4ba9-9cff-dd5172aafba9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.832575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 67/130] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:13.834392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94f8a3-2970-4ba9-9cff-dd5172aafba9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.843825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e54932-c465-48a5-be77-7321c4120097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.844849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e54932-c465-48a5-be77-7321c4120097 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.845329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e54932-c465-48a5-be77-7321c4120097 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.845979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e54932-c465-48a5-be77-7321c4120097 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add/users/57a182ce8b7e4cadb8c50cc2a7fa67ae/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.849619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e54932-c465-48a5-be77-7321c4120097 None admin] RBAC: Authorizing `identity:create_grant(project_id=812094fb94404914876df4528f073add, user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.851405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e54932-c465-48a5-be77-7321c4120097 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.929040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 64/131] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:13 2022] PUT /identity/v3/projects/812094fb94404914876df4528f073add/users/57a182ce8b7e4cadb8c50cc2a7fa67ae/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:13.930245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e54932-c465-48a5-be77-7321c4120097 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.950971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94f8a3-2970-4ba9-9cff-dd5172aafba9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.985971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.992361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8bf91-7ae7-48e5-8080-8189d2877da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:14.000239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8bf91-7ae7-48e5-8080-8189d2877da5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:14.000239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8bf91-7ae7-48e5-8080-8189d2877da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:14.000239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8bf91-7ae7-48e5-8080-8189d2877da5 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:14.000239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8bf91-7ae7-48e5-8080-8189d2877da5 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:14.000239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8bf91-7ae7-48e5-8080-8189d2877da5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:14.000239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 68/132] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:14.000684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8bf91-7ae7-48e5-8080-8189d2877da5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:14.072085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:14.076352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e54932-c465-48a5-be77-7321c4120097 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:14.119307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8bf91-7ae7-48e5-8080-8189d2877da5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:14.119523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936316b-9a1e-4155-9bd2-f1859c7d8a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:14.120191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936316b-9a1e-4155-9bd2-f1859c7d8a9e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:14.120570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936316b-9a1e-4155-9bd2-f1859c7d8a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:14.120951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936316b-9a1e-4155-9bd2-f1859c7d8a9e None admin] PATH_INFO: `/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:14.122350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936316b-9a1e-4155-9bd2-f1859c7d8a9e None admin] RBAC: Authorizing `identity:get_user(user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:14.124251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936316b-9a1e-4155-9bd2-f1859c7d8a9e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:14.126477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 65/133] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:14.127723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc0531-e86c-44d0-8c14-afd63a0315bb None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:14.128068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc0531-e86c-44d0-8c14-afd63a0315bb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:14.128407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc0531-e86c-44d0-8c14-afd63a0315bb None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:14.130208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 66/134] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:14.131001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc0531-e86c-44d0-8c14-afd63a0315bb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:14.159935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa0cec-2b19-41ae-a4c1-41ddde274168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:14.161066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa0cec-2b19-41ae-a4c1-41ddde274168 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:14.161561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa0cec-2b19-41ae-a4c1-41ddde274168 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:14.162065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa0cec-2b19-41ae-a4c1-41ddde274168 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add/users/9d4d38ef7ecd40a380aba27a2b564d42/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:14.164430 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa0cec-2b19-41ae-a4c1-41ddde274168 None admin] RBAC: Authorizing `identity:create_grant(project_id=812094fb94404914876df4528f073add, user_id=9d4d38ef7ecd40a380aba27a2b564d42, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:14.166107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa0cec-2b19-41ae-a4c1-41ddde274168 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:14.234216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 69/135] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:13 2022] PUT /identity/v3/projects/812094fb94404914876df4528f073add/users/9d4d38ef7ecd40a380aba27a2b564d42/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:14.235381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa0cec-2b19-41ae-a4c1-41ddde274168 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:14.279482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc0531-e86c-44d0-8c14-afd63a0315bb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:14.355279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f25fdd-1ca0-4e69-9457-ee74cc3ef501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:14.356332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f25fdd-1ca0-4e69-9457-ee74cc3ef501 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:14.356810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f25fdd-1ca0-4e69-9457-ee74cc3ef501 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:14.357277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f25fdd-1ca0-4e69-9457-ee74cc3ef501 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add/users/57a182ce8b7e4cadb8c50cc2a7fa67ae/roles/acfe3eeb91964f62abb1d8a84c36dccc` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:14.361396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f25fdd-1ca0-4e69-9457-ee74cc3ef501 None admin] RBAC: Authorizing `identity:create_grant(project_id=812094fb94404914876df4528f073add, user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae, role_id=acfe3eeb91964f62abb1d8a84c36dccc)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:14.363293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f25fdd-1ca0-4e69-9457-ee74cc3ef501 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:14.367243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:14.417418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa0cec-2b19-41ae-a4c1-41ddde274168 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:14.456764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6390ad1-64a1-4b0d-8506-e39359f49765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:14.457413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6390ad1-64a1-4b0d-8506-e39359f49765 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:14.458324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6390ad1-64a1-4b0d-8506-e39359f49765 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:14.458821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6390ad1-64a1-4b0d-8506-e39359f49765 None admin] PATH_INFO: `/v3/projects/53e1686defa64da4a51a16f90ce83c39` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:14.459871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 67/136] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:14 2022] PUT /identity/v3/projects/812094fb94404914876df4528f073add/users/57a182ce8b7e4cadb8c50cc2a7fa67ae/roles/acfe3eeb91964f62abb1d8a84c36dccc => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:14.460057 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6390ad1-64a1-4b0d-8506-e39359f49765 None admin] RBAC: Authorizing `identity:get_project(project_id=53e1686defa64da4a51a16f90ce83c39)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:14.461691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994f860-b038-4bd6-93fa-8efe1bfbfaea None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:14.461928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6390ad1-64a1-4b0d-8506-e39359f49765 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:14.462643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994f860-b038-4bd6-93fa-8efe1bfbfaea None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:14.463121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994f860-b038-4bd6-93fa-8efe1bfbfaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:14.464910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 70/137] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:14 2022] GET /identity/v3/projects/53e1686defa64da4a51a16f90ce83c39 => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:14.471044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6390ad1-64a1-4b0d-8506-e39359f49765 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:14.500879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7994f860-b038-4bd6-93fa-8efe1bfbfaea None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:14.502975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7994f860-b038-4bd6-93fa-8efe1bfbfaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:14.548252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:14.647607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 68/138] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:14.672764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6390ad1-64a1-4b0d-8506-e39359f49765 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:14.702107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:14.715238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b0a30-cbf9-47ec-8204-9ce7bba44d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:14.716243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b0a30-cbf9-47ec-8204-9ce7bba44d21 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:14.716680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b0a30-cbf9-47ec-8204-9ce7bba44d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:14.717111 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b0a30-cbf9-47ec-8204-9ce7bba44d21 None admin] PATH_INFO: `/v3/projects/53e1686defa64da4a51a16f90ce83c39/users/57a182ce8b7e4cadb8c50cc2a7fa67ae/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:14.720407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b0a30-cbf9-47ec-8204-9ce7bba44d21 None admin] RBAC: Authorizing `identity:create_grant(project_id=53e1686defa64da4a51a16f90ce83c39, user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:14.721040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b0a30-cbf9-47ec-8204-9ce7bba44d21 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:14.807343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 71/139] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:14 2022] PUT /identity/v3/projects/53e1686defa64da4a51a16f90ce83c39/users/57a182ce8b7e4cadb8c50cc2a7fa67ae/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:12:14.998197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994f860-b038-4bd6-93fa-8efe1bfbfaea None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:15.097443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646924a-2e27-435a-b500-e2b267eacda2 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:15.098104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646924a-2e27-435a-b500-e2b267eacda2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:15.099235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646924a-2e27-435a-b500-e2b267eacda2 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:15.101025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 72/140] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:12:15.107592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d24bfc-0e63-4d69-a25a-9fb29b53183e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:15.107873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d24bfc-0e63-4d69-a25a-9fb29b53183e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:15.108141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d24bfc-0e63-4d69-a25a-9fb29b53183e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:15.128002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994f860-b038-4bd6-93fa-8efe1bfbfaea None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:15.130587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-78d24bfc-0e63-4d69-a25a-9fb29b53183e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:15.132004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-78d24bfc-0e63-4d69-a25a-9fb29b53183e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:15.193229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fb368-c0a5-4358-b326-c6685405c249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:15.194411 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fb368-c0a5-4358-b326-c6685405c249 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:15.194895 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fb368-c0a5-4358-b326-c6685405c249 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:15.195361 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fb368-c0a5-4358-b326-c6685405c249 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:15.196694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fb368-c0a5-4358-b326-c6685405c249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:15.198681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fb368-c0a5-4358-b326-c6685405c249 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:15.201205 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 69/141] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:15.210003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fb368-c0a5-4358-b326-c6685405c249 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:15.240125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 73/142] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:15.255660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:15.352225 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fb368-c0a5-4358-b326-c6685405c249 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:15.398630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a114fa-eb5a-4c2f-a128-bb808cb745f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:15.399410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a114fa-eb5a-4c2f-a128-bb808cb745f0 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:15.399862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a114fa-eb5a-4c2f-a128-bb808cb745f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:15.400245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a114fa-eb5a-4c2f-a128-bb808cb745f0 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:15.401082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a114fa-eb5a-4c2f-a128-bb808cb745f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:15.402729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a114fa-eb5a-4c2f-a128-bb808cb745f0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:15.526163 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 70/143] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:15 2022] POST /identity/v3/users => generated 283 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:12:15.684574 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d24bfc-0e63-4d69-a25a-9fb29b53183e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:15.826831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d24bfc-0e63-4d69-a25a-9fb29b53183e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:15.845960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:15.869232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da68a8f-2158-4e2b-b9b9-0dfd752a26bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:15.869933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da68a8f-2158-4e2b-b9b9-0dfd752a26bc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:15.870211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da68a8f-2158-4e2b-b9b9-0dfd752a26bc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:15.870482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da68a8f-2158-4e2b-b9b9-0dfd752a26bc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:15.888854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da68a8f-2158-4e2b-b9b9-0dfd752a26bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5da68a8f-2158-4e2b-b9b9-0dfd752a26bc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:15.893930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:15.895398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:15.896615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 74/144] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:15.901668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a114fa-eb5a-4c2f-a128-bb808cb745f0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:15.999146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:16.018843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a114fa-eb5a-4c2f-a128-bb808cb745f0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:16.061211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9157bde-d0b4-4b39-9551-a1b832f21886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:16.062012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9157bde-d0b4-4b39-9551-a1b832f21886 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:16.062302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9157bde-d0b4-4b39-9551-a1b832f21886 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:16.062578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9157bde-d0b4-4b39-9551-a1b832f21886 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:16.063244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9157bde-d0b4-4b39-9551-a1b832f21886 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:16.063550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9157bde-d0b4-4b39-9551-a1b832f21886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:16.064945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9157bde-d0b4-4b39-9551-a1b832f21886 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:16.077274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 71/145] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:16.083332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da68a8f-2158-4e2b-b9b9-0dfd752a26bc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:16.221780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da68a8f-2158-4e2b-b9b9-0dfd752a26bc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:16.269306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cf5c5-1615-485c-af3c-0d5d7a101a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:16.269899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf5c5-1615-485c-af3c-0d5d7a101a3b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:16.270241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf5c5-1615-485c-af3c-0d5d7a101a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:16.271122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf5c5-1615-485c-af3c-0d5d7a101a3b None admin] PATH_INFO: `/v3/users/a2d376e28f7f47888a22de4d938c2cfe` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:16.273276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cf5c5-1615-485c-af3c-0d5d7a101a3b None admin] RBAC: Authorizing `identity:get_user(user_id=a2d376e28f7f47888a22de4d938c2cfe)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:16.274811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cf5c5-1615-485c-af3c-0d5d7a101a3b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:16.276510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 75/146] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:16 2022] GET /identity/v3/users/a2d376e28f7f47888a22de4d938c2cfe => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:16.282401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9157bde-d0b4-4b39-9551-a1b832f21886 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:16.453072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9157bde-d0b4-4b39-9551-a1b832f21886 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:16.493122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28530c97-81b5-4e4e-a706-db3aef667a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:16.493791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28530c97-81b5-4e4e-a706-db3aef667a2f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:16.494066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28530c97-81b5-4e4e-a706-db3aef667a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:16.494334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28530c97-81b5-4e4e-a706-db3aef667a2f None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:16.496084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28530c97-81b5-4e4e-a706-db3aef667a2f None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:16.497588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28530c97-81b5-4e4e-a706-db3aef667a2f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:16.499212 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 72/147] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:16 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:16.510643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cf5c5-1615-485c-af3c-0d5d7a101a3b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:16.572834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:16.588501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fe8ca-27c8-4584-b3c0-4dd2e11287ed None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:16.588989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fe8ca-27c8-4584-b3c0-4dd2e11287ed None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:16.589061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fe8ca-27c8-4584-b3c0-4dd2e11287ed None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:16.590631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 73/148] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:16.595816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8753ac-2de1-4556-86bb-2d9e7512abda None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:16.596092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8753ac-2de1-4556-86bb-2d9e7512abda None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:16.596370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8753ac-2de1-4556-86bb-2d9e7512abda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:16.619877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5e8753ac-2de1-4556-86bb-2d9e7512abda None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:16.621087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5e8753ac-2de1-4556-86bb-2d9e7512abda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:16.630450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:16.650976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cf5c5-1615-485c-af3c-0d5d7a101a3b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:16.720856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d155ca-31a1-4b9d-9e96-01864508ae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:16.721614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d155ca-31a1-4b9d-9e96-01864508ae90 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:16.721891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d155ca-31a1-4b9d-9e96-01864508ae90 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:16.722163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d155ca-31a1-4b9d-9e96-01864508ae90 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add/users/a2d376e28f7f47888a22de4d938c2cfe/roles/d529d6d058424562a72162b0c257851f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:16.725446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d155ca-31a1-4b9d-9e96-01864508ae90 None admin] RBAC: Authorizing `identity:create_grant(project_id=812094fb94404914876df4528f073add, user_id=a2d376e28f7f47888a22de4d938c2cfe, role_id=d529d6d058424562a72162b0c257851f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:16.726254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 74/149] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:16.727455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d155ca-31a1-4b9d-9e96-01864508ae90 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:16.812554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 76/150] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:16 2022] PUT /identity/v3/projects/812094fb94404914876df4528f073add/users/a2d376e28f7f47888a22de4d938c2cfe/roles/d529d6d058424562a72162b0c257851f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:17.108131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2730f4-4fc6-48ec-9ce4-7c6fac4dd01c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:17.108512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2730f4-4fc6-48ec-9ce4-7c6fac4dd01c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:17.108714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2730f4-4fc6-48ec-9ce4-7c6fac4dd01c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:17.110589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 75/151] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:17.116436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de122087-7333-4822-a5b0-5dc94c5ddff4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:17.116714 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de122087-7333-4822-a5b0-5dc94c5ddff4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:17.116996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de122087-7333-4822-a5b0-5dc94c5ddff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:17.140100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-de122087-7333-4822-a5b0-5dc94c5ddff4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:17.141304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-de122087-7333-4822-a5b0-5dc94c5ddff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:17.232061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 77/152] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:17.342564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2730f4-4fc6-48ec-9ce4-7c6fac4dd01c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:17.462561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2730f4-4fc6-48ec-9ce4-7c6fac4dd01c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:17.505114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92367098-93a6-4a20-8e4f-7ed10d132057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:17.505860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367098-93a6-4a20-8e4f-7ed10d132057 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:17.506137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367098-93a6-4a20-8e4f-7ed10d132057 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:17.506408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367098-93a6-4a20-8e4f-7ed10d132057 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:17.520906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92367098-93a6-4a20-8e4f-7ed10d132057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-92367098-93a6-4a20-8e4f-7ed10d132057 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:17.525879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:17.526721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:17.527478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 76/153] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:17.532704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de122087-7333-4822-a5b0-5dc94c5ddff4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:17.574124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:17.584020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848da6a-1f72-4c2a-9f88-c1f182ae063b None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:17.584321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848da6a-1f72-4c2a-9f88-c1f182ae063b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:17.584578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848da6a-1f72-4c2a-9f88-c1f182ae063b None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:17.586722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 77/154] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:12:17.592513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae59df1-95c1-4604-b9e0-408b321a5cb2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:17.593068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae59df1-95c1-4604-b9e0-408b321a5cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:17.593363 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae59df1-95c1-4604-b9e0-408b321a5cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:17.603548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:17.630104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-dae59df1-95c1-4604-b9e0-408b321a5cb2 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:17.631486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dae59df1-95c1-4604-b9e0-408b321a5cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:17.691798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de122087-7333-4822-a5b0-5dc94c5ddff4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:17.750462 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac5c41-1b78-441f-86c6-6ebecff7ab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:17.751102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac5c41-1b78-441f-86c6-6ebecff7ab94 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:17.751359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac5c41-1b78-441f-86c6-6ebecff7ab94 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:17.757493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac5c41-1b78-441f-86c6-6ebecff7ab94 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:17.757493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac5c41-1b78-441f-86c6-6ebecff7ab94 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:17.757493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac5c41-1b78-441f-86c6-6ebecff7ab94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:17.759482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac5c41-1b78-441f-86c6-6ebecff7ab94 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:17.787677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 78/155] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:17.791269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac5c41-1b78-441f-86c6-6ebecff7ab94 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:17.837713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 78/156] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:17.837713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae59df1-95c1-4604-b9e0-408b321a5cb2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:17.993572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae59df1-95c1-4604-b9e0-408b321a5cb2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:18.038936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:18.054152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8dcda-717a-4769-a471-c6cd639bf2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:18.054807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8dcda-717a-4769-a471-c6cd639bf2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.055108 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8dcda-717a-4769-a471-c6cd639bf2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.055398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8dcda-717a-4769-a471-c6cd639bf2c2 None admin] PATH_INFO: `/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.056634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8dcda-717a-4769-a471-c6cd639bf2c2 None admin] RBAC: Authorizing `identity:get_user(user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:18.058152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8dcda-717a-4769-a471-c6cd639bf2c2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:18.059751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 79/157] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:17 2022] GET /identity/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:18.060102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:18.062127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9bf1e-aec0-4e79-9a4c-bfdcc4570f79 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.062411 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9bf1e-aec0-4e79-9a4c-bfdcc4570f79 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.062686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9bf1e-aec0-4e79-9a4c-bfdcc4570f79 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.063916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 80/158] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:18.066541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9bf1e-aec0-4e79-9a4c-bfdcc4570f79 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:18.071822 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac5c41-1b78-441f-86c6-6ebecff7ab94 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:18.154554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cf42b-646f-4639-94e2-bb756f8ca56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:18.155594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cf42b-646f-4639-94e2-bb756f8ca56e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.156077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cf42b-646f-4639-94e2-bb756f8ca56e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.156539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cf42b-646f-4639-94e2-bb756f8ca56e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.180918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81cf42b-646f-4639-94e2-bb756f8ca56e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c81cf42b-646f-4639-94e2-bb756f8ca56e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:18.186605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:18.187507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:18.188280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:18.190546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 79/159] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:18.198839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c7f27-c8cb-4342-86b9-0c9c04023100 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.199391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c7f27-c8cb-4342-86b9-0c9c04023100 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.200120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c7f27-c8cb-4342-86b9-0c9c04023100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.226671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-374c7f27-c8cb-4342-86b9-0c9c04023100 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:18.228228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-374c7f27-c8cb-4342-86b9-0c9c04023100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:18.238908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9bf1e-aec0-4e79-9a4c-bfdcc4570f79 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:18.328054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bb0f1-c9ca-45c5-8ce5-f9108e76eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:18.328639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bb0f1-c9ca-45c5-8ce5-f9108e76eddc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.328924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bb0f1-c9ca-45c5-8ce5-f9108e76eddc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.329197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bb0f1-c9ca-45c5-8ce5-f9108e76eddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.339697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 80/160] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:18.340799 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c7f27-c8cb-4342-86b9-0c9c04023100 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:18.477819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:18.488134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c7f27-c8cb-4342-86b9-0c9c04023100 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:18.503897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bb0f1-c9ca-45c5-8ce5-f9108e76eddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:18.508217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bb0f1-c9ca-45c5-8ce5-f9108e76eddc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:18.508829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:18.586716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b62ea-4641-41e5-a6b8-855bb4020e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:18.587343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b62ea-4641-41e5-a6b8-855bb4020e93 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.587816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b62ea-4641-41e5-a6b8-855bb4020e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.588250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b62ea-4641-41e5-a6b8-855bb4020e93 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.589067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b62ea-4641-41e5-a6b8-855bb4020e93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:18.589610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b62ea-4641-41e5-a6b8-855bb4020e93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:18.591121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b62ea-4641-41e5-a6b8-855bb4020e93 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:18.608987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 81/161] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:18.611094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b62ea-4641-41e5-a6b8-855bb4020e93 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:18.659240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 81/162] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:18.660105 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bb0f1-c9ca-45c5-8ce5-f9108e76eddc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:18.733190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b62ea-4641-41e5-a6b8-855bb4020e93 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:18.769775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4889d-1d2d-4a23-baf8-178081842339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:18.771660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4889d-1d2d-4a23-baf8-178081842339 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.772083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4889d-1d2d-4a23-baf8-178081842339 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.772496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4889d-1d2d-4a23-baf8-178081842339 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.783712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4889d-1d2d-4a23-baf8-178081842339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-adf4889d-1d2d-4a23-baf8-178081842339 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:18.789789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:18.790868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:18.791769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:18.793790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 82/163] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:18.794833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4889d-1d2d-4a23-baf8-178081842339 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:18.807306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:18.838701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:18.843667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bb0f1-c9ca-45c5-8ce5-f9108e76eddc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:18.907015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a0c53-9480-41cf-8976-80e308a5d4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:18.907678 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4889d-1d2d-4a23-baf8-178081842339 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:18.908044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a0c53-9480-41cf-8976-80e308a5d4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.908477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a0c53-9480-41cf-8976-80e308a5d4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.908894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a0c53-9480-41cf-8976-80e308a5d4e5 None admin] PATH_INFO: `/v3/users/9d4d38ef7ecd40a380aba27a2b564d42` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.910335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a0c53-9480-41cf-8976-80e308a5d4e5 None admin] RBAC: Authorizing `identity:get_user(user_id=9d4d38ef7ecd40a380aba27a2b564d42)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:18.912018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a0c53-9480-41cf-8976-80e308a5d4e5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:18.913774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 82/164] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:18 2022] GET /identity/v3/users/9d4d38ef7ecd40a380aba27a2b564d42 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:18.914663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a0c53-9480-41cf-8976-80e308a5d4e5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:18.950858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2d04d-0bb0-424f-aef3-7cfebea07fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:18.951894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2d04d-0bb0-424f-aef3-7cfebea07fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.952395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2d04d-0bb0-424f-aef3-7cfebea07fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.952830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2d04d-0bb0-424f-aef3-7cfebea07fb0 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.954137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2d04d-0bb0-424f-aef3-7cfebea07fb0 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:18.955729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2d04d-0bb0-424f-aef3-7cfebea07fb0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:18.957563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 83/165] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:18 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:18.958538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2d04d-0bb0-424f-aef3-7cfebea07fb0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:19.053585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:19.094447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2d04d-0bb0-424f-aef3-7cfebea07fb0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:19.096418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a0c53-9480-41cf-8976-80e308a5d4e5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:19.170515 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a6509-5f8a-4d66-abf1-9c69377c97c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:19.171139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a6509-5f8a-4d66-abf1-9c69377c97c9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:19.171422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a6509-5f8a-4d66-abf1-9c69377c97c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:19.171688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a6509-5f8a-4d66-abf1-9c69377c97c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:19.172359 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a6509-5f8a-4d66-abf1-9c69377c97c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:19.172660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a6509-5f8a-4d66-abf1-9c69377c97c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:19.174146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a6509-5f8a-4d66-abf1-9c69377c97c9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:19.175041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc69b-793f-4f2a-98e0-9b959528ebe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:19.175776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc69b-793f-4f2a-98e0-9b959528ebe9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:19.176062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc69b-793f-4f2a-98e0-9b959528ebe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:19.176395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc69b-793f-4f2a-98e0-9b959528ebe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:19.189588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 83/166] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:19.190445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a6509-5f8a-4d66-abf1-9c69377c97c9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:19.236250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:19.321060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fc69b-793f-4f2a-98e0-9b959528ebe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:19.322570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fc69b-793f-4f2a-98e0-9b959528ebe9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:19.412885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a6509-5f8a-4d66-abf1-9c69377c97c9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:19.477617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:19.498205 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747ced3-881d-4ed8-bc87-954d12178491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:19.500371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747ced3-881d-4ed8-bc87-954d12178491 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:19.500371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747ced3-881d-4ed8-bc87-954d12178491 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:19.500371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747ced3-881d-4ed8-bc87-954d12178491 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:19.522370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747ced3-881d-4ed8-bc87-954d12178491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:19.525474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 84/167] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:19.526320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc69b-793f-4f2a-98e0-9b959528ebe9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-4747ced3-881d-4ed8-bc87-954d12178491 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:19.527536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:19.528452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 84/168] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:19.529622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747ced3-881d-4ed8-bc87-954d12178491 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:19.657375 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc69b-793f-4f2a-98e0-9b959528ebe9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:19.735883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36df4a-4a9c-4e34-9f1d-2409053977d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:19.736728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36df4a-4a9c-4e34-9f1d-2409053977d4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:19.737189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36df4a-4a9c-4e34-9f1d-2409053977d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:19.737653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36df4a-4a9c-4e34-9f1d-2409053977d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:19.738540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36df4a-4a9c-4e34-9f1d-2409053977d4 None admin] RBAC: Adding query filter params (role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, scope.project.id=812094fb94404914876df4528f073add, user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:19.739876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:19.741354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36df4a-4a9c-4e34-9f1d-2409053977d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:19.743012 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36df4a-4a9c-4e34-9f1d-2409053977d4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:19.754481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 85/169] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:19 2022] GET /identity/v3/role_assignments?user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae&scope.project.id=812094fb94404914876df4528f073add&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:19.755636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36df4a-4a9c-4e34-9f1d-2409053977d4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:19.777879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747ced3-881d-4ed8-bc87-954d12178491 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:19.836181 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:19.894364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d80e9-dde9-4ca4-8a81-4d57a75cf727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:19.895235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d80e9-dde9-4ca4-8a81-4d57a75cf727 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:19.895559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d80e9-dde9-4ca4-8a81-4d57a75cf727 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:19.896015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d80e9-dde9-4ca4-8a81-4d57a75cf727 None admin] PATH_INFO: `/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:19.897502 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d80e9-dde9-4ca4-8a81-4d57a75cf727 None admin] RBAC: Authorizing `identity:get_user(user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:19.899012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d80e9-dde9-4ca4-8a81-4d57a75cf727 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:19.900814 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 85/170] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:19 2022] GET /identity/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae => generated 275 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:19.901406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d80e9-dde9-4ca4-8a81-4d57a75cf727 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:19.987546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36df4a-4a9c-4e34-9f1d-2409053977d4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:20.075805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6f854-953a-45ca-ae44-c83d98042ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:20.076703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6f854-953a-45ca-ae44-c83d98042ebd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.077154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6f854-953a-45ca-ae44-c83d98042ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.077607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6f854-953a-45ca-ae44-c83d98042ebd None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.081070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6f854-953a-45ca-ae44-c83d98042ebd None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:20.083299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6f854-953a-45ca-ae44-c83d98042ebd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:20.085549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 86/171] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:19 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:20.087122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e6727-a074-4ee4-8f37-8e105de93704 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.087554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e6727-a074-4ee4-8f37-8e105de93704 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.087972 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e6727-a074-4ee4-8f37-8e105de93704 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.089385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 87/172] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:20.090352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e6727-a074-4ee4-8f37-8e105de93704 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:20.121956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d80e9-dde9-4ca4-8a81-4d57a75cf727 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:20.190030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257f6c8-1023-485e-8974-31e29e7154e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:20.190645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257f6c8-1023-485e-8974-31e29e7154e9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.190922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257f6c8-1023-485e-8974-31e29e7154e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.191190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257f6c8-1023-485e-8974-31e29e7154e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.191936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257f6c8-1023-485e-8974-31e29e7154e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:20.192417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257f6c8-1023-485e-8974-31e29e7154e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:20.193853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257f6c8-1023-485e-8974-31e29e7154e9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:20.203982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 86/173] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:20.204885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257f6c8-1023-485e-8974-31e29e7154e9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:20.231129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:20.280075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e6727-a074-4ee4-8f37-8e105de93704 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:20.294900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:20.321126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257f6c8-1023-485e-8974-31e29e7154e9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:20.339734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03568d-e4cc-4c81-b936-802c8af3089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:20.340356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03568d-e4cc-4c81-b936-802c8af3089b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.340631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03568d-e4cc-4c81-b936-802c8af3089b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.340896 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03568d-e4cc-4c81-b936-802c8af3089b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.361231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade40147-ae94-41f4-917e-a985e119028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:20.361868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade40147-ae94-41f4-917e-a985e119028d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.362145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade40147-ae94-41f4-917e-a985e119028d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.362416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade40147-ae94-41f4-917e-a985e119028d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.363175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade40147-ae94-41f4-917e-a985e119028d None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=812094fb94404914876df4528f073add, user.id=9d4d38ef7ecd40a380aba27a2b564d42) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:20.363485 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade40147-ae94-41f4-917e-a985e119028d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:20.365883 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade40147-ae94-41f4-917e-a985e119028d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:20.378606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 87/174] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:20 2022] GET /identity/v3/role_assignments?user.id=9d4d38ef7ecd40a380aba27a2b564d42&scope.project.id=812094fb94404914876df4528f073add&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:20.380851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c3eff-8007-4cf7-bd87-2399a0a776d4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.381479 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c3eff-8007-4cf7-bd87-2399a0a776d4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.385480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c3eff-8007-4cf7-bd87-2399a0a776d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.412586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0d3c3eff-8007-4cf7-bd87-2399a0a776d4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:20.414245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0d3c3eff-8007-4cf7-bd87-2399a0a776d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:20.448563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03568d-e4cc-4c81-b936-802c8af3089b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:20.450356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03568d-e4cc-4c81-b936-802c8af3089b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:20.516208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 88/175] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:20.518164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c3eff-8007-4cf7-bd87-2399a0a776d4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:20.554461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:20.588551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 88/176] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:20.590232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63337e-10fc-49c3-9f74-725af6b4d632 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.590816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63337e-10fc-49c3-9f74-725af6b4d632 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.591282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63337e-10fc-49c3-9f74-725af6b4d632 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.593192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 89/177] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:20.594550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fe5d6-d933-4fd6-b770-8789f601a8b1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.595048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fe5d6-d933-4fd6-b770-8789f601a8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.595510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fe5d6-d933-4fd6-b770-8789f601a8b1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.596941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 90/178] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:20.597956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fe5d6-d933-4fd6-b770-8789f601a8b1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:20.611423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:20.661895 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c3eff-8007-4cf7-bd87-2399a0a776d4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:20.700580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee314c-82a2-43c8-bbe5-2e4c17b46dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:20.701111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee314c-82a2-43c8-bbe5-2e4c17b46dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.701394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee314c-82a2-43c8-bbe5-2e4c17b46dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.701685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee314c-82a2-43c8-bbe5-2e4c17b46dd9 None admin] PATH_INFO: `/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.702826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee314c-82a2-43c8-bbe5-2e4c17b46dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=57a182ce8b7e4cadb8c50cc2a7fa67ae)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:20.704285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee314c-82a2-43c8-bbe5-2e4c17b46dd9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:20.706881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 89/179] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:20 2022] GET /identity/v3/users/57a182ce8b7e4cadb8c50cc2a7fa67ae => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:20.708112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda29cf8-36ea-4eb9-9675-d2a25f97a19b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.708419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda29cf8-36ea-4eb9-9675-d2a25f97a19b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.708955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda29cf8-36ea-4eb9-9675-d2a25f97a19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.720245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fe5d6-d933-4fd6-b770-8789f601a8b1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:20.729939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fda29cf8-36ea-4eb9-9675-d2a25f97a19b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:20.733478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fda29cf8-36ea-4eb9-9675-d2a25f97a19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:20.759458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f858fa-8d4a-48be-9da0-76900302e588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:20.760238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f858fa-8d4a-48be-9da0-76900302e588 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.760635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f858fa-8d4a-48be-9da0-76900302e588 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.760977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f858fa-8d4a-48be-9da0-76900302e588 None admin] PATH_INFO: `/v3/projects/53e1686defa64da4a51a16f90ce83c39` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.762239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f858fa-8d4a-48be-9da0-76900302e588 None admin] RBAC: Authorizing `identity:get_project(project_id=53e1686defa64da4a51a16f90ce83c39)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:20.763815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f858fa-8d4a-48be-9da0-76900302e588 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:20.765894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 91/180] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:20 2022] GET /identity/v3/projects/53e1686defa64da4a51a16f90ce83c39 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:20.767424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba8dec-50de-47db-aeb0-63ed73ddbbac None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.767846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba8dec-50de-47db-aeb0-63ed73ddbbac None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.768213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba8dec-50de-47db-aeb0-63ed73ddbbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.789489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b4ba8dec-50de-47db-aeb0-63ed73ddbbac None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:20.790803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b4ba8dec-50de-47db-aeb0-63ed73ddbbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:20.807720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 90/181] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:20.808429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda29cf8-36ea-4eb9-9675-d2a25f97a19b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:20.829506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:20.875855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 92/182] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:20.875855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba8dec-50de-47db-aeb0-63ed73ddbbac None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:20.884999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:20.934415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda29cf8-36ea-4eb9-9675-d2a25f97a19b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:20.978999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9893c-5f15-41f3-8cd9-109349b138ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:20.979881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9893c-5f15-41f3-8cd9-109349b138ba None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:20.980383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9893c-5f15-41f3-8cd9-109349b138ba None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:20.980867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9893c-5f15-41f3-8cd9-109349b138ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:20.995872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba8dec-50de-47db-aeb0-63ed73ddbbac None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.041344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06207738-c6c2-4934-9ad2-40602e6f66e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.042324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207738-c6c2-4934-9ad2-40602e6f66e6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.042673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207738-c6c2-4934-9ad2-40602e6f66e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.043017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207738-c6c2-4934-9ad2-40602e6f66e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.044232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06207738-c6c2-4934-9ad2-40602e6f66e6 None admin] RBAC: Adding query filter params (role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, scope.project.id=53e1686defa64da4a51a16f90ce83c39, user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:21.044768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06207738-c6c2-4934-9ad2-40602e6f66e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.046637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06207738-c6c2-4934-9ad2-40602e6f66e6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.056740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 93/183] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:20 2022] GET /identity/v3/role_assignments?user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae&scope.project.id=53e1686defa64da4a51a16f90ce83c39&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:21.058315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca66b0-1497-48ae-8d9e-78c2ff023aac None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.058860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca66b0-1497-48ae-8d9e-78c2ff023aac None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.059327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca66b0-1497-48ae-8d9e-78c2ff023aac None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.060873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 94/184] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:21.061112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9893c-5f15-41f3-8cd9-109349b138ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.062120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca66b0-1497-48ae-8d9e-78c2ff023aac None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.062615 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9893c-5f15-41f3-8cd9-109349b138ba None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.082302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:21.146633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:21.193745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca66b0-1497-48ae-8d9e-78c2ff023aac None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.208010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 91/185] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:21.209293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856bfc-3042-4bb9-a05a-26df441d9450 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.209614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856bfc-3042-4bb9-a05a-26df441d9450 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.209882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856bfc-3042-4bb9-a05a-26df441d9450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.238404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd0310-c8ef-4777-b736-9ee6777b170c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.239190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd0310-c8ef-4777-b736-9ee6777b170c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.239661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd0310-c8ef-4777-b736-9ee6777b170c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.239942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd0310-c8ef-4777-b736-9ee6777b170c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.250333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd0310-c8ef-4777-b736-9ee6777b170c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-15fd0310-c8ef-4777-b736-9ee6777b170c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:21.256275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:21.257167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:21.257993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 95/186] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:21.260469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd0310-c8ef-4777-b736-9ee6777b170c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.261429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-03856bfc-3042-4bb9-a05a-26df441d9450 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:21.262661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-03856bfc-3042-4bb9-a05a-26df441d9450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:21.337704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 92/187] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:21.340026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03856bfc-3042-4bb9-a05a-26df441d9450 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.365971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd0310-c8ef-4777-b736-9ee6777b170c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.384761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:21.396941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:21.403561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b28ee-aec4-4a1e-9cb9-749b998c1673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.404187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b28ee-aec4-4a1e-9cb9-749b998c1673 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.404460 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b28ee-aec4-4a1e-9cb9-749b998c1673 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.404731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b28ee-aec4-4a1e-9cb9-749b998c1673 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.416938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b28ee-aec4-4a1e-9cb9-749b998c1673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-403b28ee-aec4-4a1e-9cb9-749b998c1673 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:21.422249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:21.423283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 96/188] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:21.424116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b28ee-aec4-4a1e-9cb9-749b998c1673 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.455443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03856bfc-3042-4bb9-a05a-26df441d9450 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.493835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4dca8-971e-4230-800d-cb1e09cb8afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.494473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4dca8-971e-4230-800d-cb1e09cb8afd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.494747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4dca8-971e-4230-800d-cb1e09cb8afd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.495010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4dca8-971e-4230-800d-cb1e09cb8afd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.506361 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4dca8-971e-4230-800d-cb1e09cb8afd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-bea4dca8-971e-4230-800d-cb1e09cb8afd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:21.511136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:21.512014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:21.512849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 93/189] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:21.513541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4dca8-971e-4230-800d-cb1e09cb8afd None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.533691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b28ee-aec4-4a1e-9cb9-749b998c1673 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.569341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018db3a4-8950-49bd-a0f8-3bce7f0dcfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.570000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018db3a4-8950-49bd-a0f8-3bce7f0dcfbb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.570278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018db3a4-8950-49bd-a0f8-3bce7f0dcfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.570547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018db3a4-8950-49bd-a0f8-3bce7f0dcfbb None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.571661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018db3a4-8950-49bd-a0f8-3bce7f0dcfbb None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.573102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018db3a4-8950-49bd-a0f8-3bce7f0dcfbb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.579687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 97/190] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:21.579687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018db3a4-8950-49bd-a0f8-3bce7f0dcfbb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.627869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4dca8-971e-4230-800d-cb1e09cb8afd None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.629303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:21.636305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:21.663036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23254d9a-ba69-4848-92f3-aa77d1c6011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.663926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23254d9a-ba69-4848-92f3-aa77d1c6011f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.664199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23254d9a-ba69-4848-92f3-aa77d1c6011f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.664467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23254d9a-ba69-4848-92f3-aa77d1c6011f None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.665122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23254d9a-ba69-4848-92f3-aa77d1c6011f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:21.665422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23254d9a-ba69-4848-92f3-aa77d1c6011f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.666838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23254d9a-ba69-4848-92f3-aa77d1c6011f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.674831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 94/191] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:21.675621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23254d9a-ba69-4848-92f3-aa77d1c6011f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.678004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018db3a4-8950-49bd-a0f8-3bce7f0dcfbb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.713375 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014d818-d1c9-43ed-9ef8-170eeceb9ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.714009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014d818-d1c9-43ed-9ef8-170eeceb9ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.714287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014d818-d1c9-43ed-9ef8-170eeceb9ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.714556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014d818-d1c9-43ed-9ef8-170eeceb9ce6 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.715216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014d818-d1c9-43ed-9ef8-170eeceb9ce6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:21.715518 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014d818-d1c9-43ed-9ef8-170eeceb9ce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.716885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014d818-d1c9-43ed-9ef8-170eeceb9ce6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.728309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 98/192] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:21.729489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014d818-d1c9-43ed-9ef8-170eeceb9ce6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.786643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23254d9a-ba69-4848-92f3-aa77d1c6011f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.820523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7eb37-4c3c-48dc-9c46-f58dbf3105fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.821131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7eb37-4c3c-48dc-9c46-f58dbf3105fe None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.821386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7eb37-4c3c-48dc-9c46-f58dbf3105fe None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.821683 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7eb37-4c3c-48dc-9c46-f58dbf3105fe None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.822339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7eb37-4c3c-48dc-9c46-f58dbf3105fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:21.822636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7eb37-4c3c-48dc-9c46-f58dbf3105fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.824007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7eb37-4c3c-48dc-9c46-f58dbf3105fe None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.832119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 95/193] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:21.833118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7eb37-4c3c-48dc-9c46-f58dbf3105fe None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.833241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014d818-d1c9-43ed-9ef8-170eeceb9ce6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.853382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:21.862797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:21.867631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b044a4-5a87-4406-8ee3-b8ab5409acae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.868245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b044a4-5a87-4406-8ee3-b8ab5409acae None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.870339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b044a4-5a87-4406-8ee3-b8ab5409acae None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.870339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b044a4-5a87-4406-8ee3-b8ab5409acae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.870339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b044a4-5a87-4406-8ee3-b8ab5409acae None admin] RBAC: Adding query filter params (role.id=acfe3eeb91964f62abb1d8a84c36dccc, scope.project.id=812094fb94404914876df4528f073add, user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:21.870339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b044a4-5a87-4406-8ee3-b8ab5409acae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.871252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b044a4-5a87-4406-8ee3-b8ab5409acae None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.876840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 99/194] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/role_assignments?user.id=57a182ce8b7e4cadb8c50cc2a7fa67ae&scope.project.id=812094fb94404914876df4528f073add&role.id=acfe3eeb91964f62abb1d8a84c36dccc => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:21.877650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b044a4-5a87-4406-8ee3-b8ab5409acae None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.951840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7eb37-4c3c-48dc-9c46-f58dbf3105fe None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.989828 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda14612-4f38-49e6-88d0-380ca51df067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.990469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda14612-4f38-49e6-88d0-380ca51df067 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.990742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda14612-4f38-49e6-88d0-380ca51df067 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.991026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda14612-4f38-49e6-88d0-380ca51df067 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.992172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda14612-4f38-49e6-88d0-380ca51df067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.993666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda14612-4f38-49e6-88d0-380ca51df067 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.994736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b044a4-5a87-4406-8ee3-b8ab5409acae None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.995450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 96/195] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:22.000836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda14612-4f38-49e6-88d0-380ca51df067 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:22.031329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdaa5c-d4f1-4704-a537-ee149e090e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:22.032259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdaa5c-d4f1-4704-a537-ee149e090e79 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:22.032533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdaa5c-d4f1-4704-a537-ee149e090e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:22.032809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdaa5c-d4f1-4704-a537-ee149e090e79 None admin] PATH_INFO: `/v3/users/33f498cc19a44628933942cd4aaad2d4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:22.053693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdaa5c-d4f1-4704-a537-ee149e090e79 None admin] RBAC: Authorizing `identity:get_user(user_id=33f498cc19a44628933942cd4aaad2d4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:22.056411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdaa5c-d4f1-4704-a537-ee149e090e79 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:22.058487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 100/196] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/users/33f498cc19a44628933942cd4aaad2d4 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:22.059366 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdaa5c-d4f1-4704-a537-ee149e090e79 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:22.108351 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:22.113024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:22.120847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda14612-4f38-49e6-88d0-380ca51df067 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:22.154881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283c97c-1b0a-4822-acc3-eaae7d8cbf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:22.155511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283c97c-1b0a-4822-acc3-eaae7d8cbf93 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:22.155792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283c97c-1b0a-4822-acc3-eaae7d8cbf93 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:22.156067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283c97c-1b0a-4822-acc3-eaae7d8cbf93 None admin] PATH_INFO: `/v3/users/33f498cc19a44628933942cd4aaad2d4` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:22.157231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283c97c-1b0a-4822-acc3-eaae7d8cbf93 None admin] RBAC: Authorizing `identity:get_user(user_id=33f498cc19a44628933942cd4aaad2d4)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:22.158728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283c97c-1b0a-4822-acc3-eaae7d8cbf93 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:22.160312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 97/197] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/users/33f498cc19a44628933942cd4aaad2d4 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:22.161651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283c97c-1b0a-4822-acc3-eaae7d8cbf93 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:22.168165 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdaa5c-d4f1-4704-a537-ee149e090e79 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:22.203847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524cf8-5843-4d4b-b61c-0f78472ca305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:22.204410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524cf8-5843-4d4b-b61c-0f78472ca305 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:22.204691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524cf8-5843-4d4b-b61c-0f78472ca305 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:22.204963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524cf8-5843-4d4b-b61c-0f78472ca305 None admin] PATH_INFO: `/v3/users/a2d376e28f7f47888a22de4d938c2cfe` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:22.206162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524cf8-5843-4d4b-b61c-0f78472ca305 None admin] RBAC: Authorizing `identity:get_user(user_id=a2d376e28f7f47888a22de4d938c2cfe)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:22.207674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524cf8-5843-4d4b-b61c-0f78472ca305 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:22.209344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 101/198] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:22 2022] GET /identity/v3/users/a2d376e28f7f47888a22de4d938c2cfe => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:22.209344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524cf8-5843-4d4b-b61c-0f78472ca305 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:22.268018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283c97c-1b0a-4822-acc3-eaae7d8cbf93 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:22.302963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac0df5-f586-4ccd-b769-347e16623091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:22.303561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac0df5-f586-4ccd-b769-347e16623091 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:22.303832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac0df5-f586-4ccd-b769-347e16623091 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:22.304101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac0df5-f586-4ccd-b769-347e16623091 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:22.304741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac0df5-f586-4ccd-b769-347e16623091 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:22.306295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac0df5-f586-4ccd-b769-347e16623091 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:22.311630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524cf8-5843-4d4b-b61c-0f78472ca305 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:22.322621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:22.345283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debffa4c-ed7b-46d0-be09-1ecaad837dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:22.345926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debffa4c-ed7b-46d0-be09-1ecaad837dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:22.346211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debffa4c-ed7b-46d0-be09-1ecaad837dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:22.346485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debffa4c-ed7b-46d0-be09-1ecaad837dc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:22.413282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debffa4c-ed7b-46d0-be09-1ecaad837dc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:22.414795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debffa4c-ed7b-46d0-be09-1ecaad837dc2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:22.422073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 98/199] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:22 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:22.422958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac0df5-f586-4ccd-b769-347e16623091 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:22.461674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:22.529102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 102/200] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:22.530031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debffa4c-ed7b-46d0-be09-1ecaad837dc2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:22.545361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:22.559091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac0df5-f586-4ccd-b769-347e16623091 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:22.602966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8fd56-2a46-4133-86d5-38b86e529d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:22.603885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8fd56-2a46-4133-86d5-38b86e529d1b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:22.604169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8fd56-2a46-4133-86d5-38b86e529d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:22.604460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8fd56-2a46-4133-86d5-38b86e529d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:22.652507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debffa4c-ed7b-46d0-be09-1ecaad837dc2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:22.691383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8fd56-2a46-4133-86d5-38b86e529d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:22.692974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8fd56-2a46-4133-86d5-38b86e529d1b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:22.701132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4d987-3f71-4143-ba64-b548124d7af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:22.701791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4d987-3f71-4143-ba64-b548124d7af9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:22.702071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4d987-3f71-4143-ba64-b548124d7af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:22.702343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4d987-3f71-4143-ba64-b548124d7af9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:22.719204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:22.794676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4d987-3f71-4143-ba64-b548124d7af9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:22.796179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4d987-3f71-4143-ba64-b548124d7af9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:22.811354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:22.822414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 99/201] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:22.823267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8fd56-2a46-4133-86d5-38b86e529d1b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:22.919798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 103/202] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:22.920852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4d987-3f71-4143-ba64-b548124d7af9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:22.936778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8fd56-2a46-4133-86d5-38b86e529d1b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:22.955931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:22.988656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bb183-4ee0-40de-9a55-648f41a598ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:22.988656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb183-4ee0-40de-9a55-648f41a598ec None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:22.988656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb183-4ee0-40de-9a55-648f41a598ec None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:22.988656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb183-4ee0-40de-9a55-648f41a598ec None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:22.989241 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bb183-4ee0-40de-9a55-648f41a598ec None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:22.989241 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bb183-4ee0-40de-9a55-648f41a598ec None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:22.989241 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 100/203] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:22 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:22.989241 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bb183-4ee0-40de-9a55-648f41a598ec None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:23.038157 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4d987-3f71-4143-ba64-b548124d7af9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:23.049977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:23.076599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b1bcb-e733-4ddb-85c7-a5fd806084d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:23.077447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b1bcb-e733-4ddb-85c7-a5fd806084d2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:23.077823 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b1bcb-e733-4ddb-85c7-a5fd806084d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:23.078243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b1bcb-e733-4ddb-85c7-a5fd806084d2 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:23.079572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b1bcb-e733-4ddb-85c7-a5fd806084d2 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:23.081562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b1bcb-e733-4ddb-85c7-a5fd806084d2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:23.083753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 104/204] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:22 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:23.084606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b1bcb-e733-4ddb-85c7-a5fd806084d2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:23.104808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bb183-4ee0-40de-9a55-648f41a598ec None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:23.143058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10ca26-2aa8-4b48-b0ef-c28ebfb77d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:23.143940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10ca26-2aa8-4b48-b0ef-c28ebfb77d27 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:23.144215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10ca26-2aa8-4b48-b0ef-c28ebfb77d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:23.145480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10ca26-2aa8-4b48-b0ef-c28ebfb77d27 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:23.146801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10ca26-2aa8-4b48-b0ef-c28ebfb77d27 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:23.148766 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10ca26-2aa8-4b48-b0ef-c28ebfb77d27 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:23.151230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 101/205] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:22 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:23.152065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10ca26-2aa8-4b48-b0ef-c28ebfb77d27 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:23.211866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b1bcb-e733-4ddb-85c7-a5fd806084d2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:23.219061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:23.261215 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717d1a5-156b-4197-9da6-b1ade8b4d13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:23.262080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717d1a5-156b-4197-9da6-b1ade8b4d13c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:23.262359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717d1a5-156b-4197-9da6-b1ade8b4d13c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:23.262627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717d1a5-156b-4197-9da6-b1ade8b4d13c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:23.263676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717d1a5-156b-4197-9da6-b1ade8b4d13c None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=d61e52cca9584059bf484cabd52d2808, user.id=33f498cc19a44628933942cd4aaad2d4) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:23.263981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717d1a5-156b-4197-9da6-b1ade8b4d13c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:23.265630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717d1a5-156b-4197-9da6-b1ade8b4d13c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:23.273295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 105/206] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:23 2022] GET /identity/v3/role_assignments?user.id=33f498cc19a44628933942cd4aaad2d4&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:23.274138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717d1a5-156b-4197-9da6-b1ade8b4d13c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:23.280805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10ca26-2aa8-4b48-b0ef-c28ebfb77d27 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:23.328567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:23.365824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5d039-9178-46aa-9ab9-5e07b24b32a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:23.366506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5d039-9178-46aa-9ab9-5e07b24b32a1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:23.366789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5d039-9178-46aa-9ab9-5e07b24b32a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:23.367065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5d039-9178-46aa-9ab9-5e07b24b32a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:23.367806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5d039-9178-46aa-9ab9-5e07b24b32a1 None admin] RBAC: Adding query filter params (role.id=acfe3eeb91964f62abb1d8a84c36dccc, scope.project.id=d61e52cca9584059bf484cabd52d2808, user.id=33f498cc19a44628933942cd4aaad2d4) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:23.368114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5d039-9178-46aa-9ab9-5e07b24b32a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:23.369603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5d039-9178-46aa-9ab9-5e07b24b32a1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:23.384838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 102/207] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:23 2022] GET /identity/v3/role_assignments?user.id=33f498cc19a44628933942cd4aaad2d4&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=acfe3eeb91964f62abb1d8a84c36dccc => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:23.394221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717d1a5-156b-4197-9da6-b1ade8b4d13c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:23.440894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bfec2-f867-45ae-ace4-fb36c7943150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:23.441549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bfec2-f867-45ae-ace4-fb36c7943150 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:23.442402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bfec2-f867-45ae-ace4-fb36c7943150 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:23.442817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bfec2-f867-45ae-ace4-fb36c7943150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:23.443846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bfec2-f867-45ae-ace4-fb36c7943150 None admin] RBAC: Adding query filter params (role.id=d529d6d058424562a72162b0c257851f, scope.project.id=812094fb94404914876df4528f073add, user.id=a2d376e28f7f47888a22de4d938c2cfe) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:23.444154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bfec2-f867-45ae-ace4-fb36c7943150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:23.445643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bfec2-f867-45ae-ace4-fb36c7943150 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:23.452861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 106/208] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:23 2022] GET /identity/v3/role_assignments?user.id=a2d376e28f7f47888a22de4d938c2cfe&scope.project.id=812094fb94404914876df4528f073add&role.id=d529d6d058424562a72162b0c257851f => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:24.612438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf59b8-21a5-48e1-8a56-8ffe70fba308 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:24.613002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf59b8-21a5-48e1-8a56-8ffe70fba308 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:24.613002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf59b8-21a5-48e1-8a56-8ffe70fba308 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:24.614556 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 103/209] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:24.619818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38258618-1161-4c71-9785-83996489b5b3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:24.620653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38258618-1161-4c71-9785-83996489b5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:24.621135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38258618-1161-4c71-9785-83996489b5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:24.634377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061a2df-2232-41bc-aca1-13d5e0e84ca2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:24.634981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061a2df-2232-41bc-aca1-13d5e0e84ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:24.635510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061a2df-2232-41bc-aca1-13d5e0e84ca2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:24.637276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 104/210] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:12:24.641062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-38258618-1161-4c71-9785-83996489b5b3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:24.642662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-38258618-1161-4c71-9785-83996489b5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:24.646169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18acdef-2581-4489-999f-bedea058fa3c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:24.646941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18acdef-2581-4489-999f-bedea058fa3c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:24.647396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18acdef-2581-4489-999f-bedea058fa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:24.667430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b18acdef-2581-4489-999f-bedea058fa3c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:24.668875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b18acdef-2581-4489-999f-bedea058fa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:24.731834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 107/211] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:24.765558 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 105/212] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:25.004678 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38258618-1161-4c71-9785-83996489b5b3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:25.031095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18acdef-2581-4489-999f-bedea058fa3c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:25.041685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:25.084836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:25.110417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38258618-1161-4c71-9785-83996489b5b3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:25.138419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18acdef-2581-4489-999f-bedea058fa3c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:25.147251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd33fb0-b7c5-4b1e-b9e4-bf2e25d4f071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:25.147906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd33fb0-b7c5-4b1e-b9e4-bf2e25d4f071 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.148473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd33fb0-b7c5-4b1e-b9e4-bf2e25d4f071 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.148473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd33fb0-b7c5-4b1e-b9e4-bf2e25d4f071 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.149575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd33fb0-b7c5-4b1e-b9e4-bf2e25d4f071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:25.151052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd33fb0-b7c5-4b1e-b9e4-bf2e25d4f071 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:25.152900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 108/213] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:25.158179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd33fb0-b7c5-4b1e-b9e4-bf2e25d4f071 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:25.178145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dd82a-8c80-4772-920d-5ec5ddc6cbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:25.179101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dd82a-8c80-4772-920d-5ec5ddc6cbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.179567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dd82a-8c80-4772-920d-5ec5ddc6cbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.180054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dd82a-8c80-4772-920d-5ec5ddc6cbf1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.192182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dd82a-8c80-4772-920d-5ec5ddc6cbf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-0e6dd82a-8c80-4772-920d-5ec5ddc6cbf1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:25.197852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:25.198789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:25.199598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:25.207215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 106/214] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:25.209369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dd82a-8c80-4772-920d-5ec5ddc6cbf1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:25.263821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd33fb0-b7c5-4b1e-b9e4-bf2e25d4f071 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:25.267028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:25.306895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3706b4-bb92-4f42-b9dc-33bb935eab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:25.308420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3706b4-bb92-4f42-b9dc-33bb935eab93 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.309247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3706b4-bb92-4f42-b9dc-33bb935eab93 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.309779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3706b4-bb92-4f42-b9dc-33bb935eab93 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.310947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3706b4-bb92-4f42-b9dc-33bb935eab93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:25.313258 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3706b4-bb92-4f42-b9dc-33bb935eab93 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:25.359121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dd82a-8c80-4772-920d-5ec5ddc6cbf1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:25.370035 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:25.394052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 109/215] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:25 2022] POST /identity/v3/users => generated 297 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:12:25.401643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49a527-1e94-4f26-8937-d9cf3a7d6278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:25.401643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49a527-1e94-4f26-8937-d9cf3a7d6278 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.401643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49a527-1e94-4f26-8937-d9cf3a7d6278 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.401643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49a527-1e94-4f26-8937-d9cf3a7d6278 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.402044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49a527-1e94-4f26-8937-d9cf3a7d6278 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:25.402044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49a527-1e94-4f26-8937-d9cf3a7d6278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:25.402044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49a527-1e94-4f26-8937-d9cf3a7d6278 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:25.403437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a286067-0839-418a-b4d9-75f1ad7af2b3 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.403776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a286067-0839-418a-b4d9-75f1ad7af2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.404486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a286067-0839-418a-b4d9-75f1ad7af2b3 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.411327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 110/216] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:25.413113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be72a9-339f-4566-b13f-9aba5c640029 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.413403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be72a9-339f-4566-b13f-9aba5c640029 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.413708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be72a9-339f-4566-b13f-9aba5c640029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.414701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 107/217] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:25.420811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49a527-1e94-4f26-8937-d9cf3a7d6278 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:25.439102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-50be72a9-339f-4566-b13f-9aba5c640029 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:25.440345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-50be72a9-339f-4566-b13f-9aba5c640029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:25.516944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 111/218] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:25.521694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f5e23-ece8-435d-a77e-358cba08ab2c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.522075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f5e23-ece8-435d-a77e-358cba08ab2c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.522412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f5e23-ece8-435d-a77e-358cba08ab2c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.523834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 112/219] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:25.529415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e9b57-f470-4ba9-9ee3-178c57344d05 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.529723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e9b57-f470-4ba9-9ee3-178c57344d05 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.529991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e9b57-f470-4ba9-9ee3-178c57344d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.549388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d25e9b57-f470-4ba9-9ee3-178c57344d05 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:25.550796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d25e9b57-f470-4ba9-9ee3-178c57344d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:25.562664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49a527-1e94-4f26-8937-d9cf3a7d6278 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:25.607240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974c358-4290-45f1-8ecb-8a6198ce41a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:25.607511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974c358-4290-45f1-8ecb-8a6198ce41a0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.607748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974c358-4290-45f1-8ecb-8a6198ce41a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.608143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974c358-4290-45f1-8ecb-8a6198ce41a0 None admin] PATH_INFO: `/v3/users/f461d4cb769f4b658b434a594e78c32c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.625947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974c358-4290-45f1-8ecb-8a6198ce41a0 None admin] RBAC: Authorizing `identity:get_user(user_id=f461d4cb769f4b658b434a594e78c32c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:25.627419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974c358-4290-45f1-8ecb-8a6198ce41a0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:25.633965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 108/220] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:25 2022] GET /identity/v3/users/f461d4cb769f4b658b434a594e78c32c => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:25.636375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974c358-4290-45f1-8ecb-8a6198ce41a0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:25.654327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 113/221] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:25.673441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:25.763513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974c358-4290-45f1-8ecb-8a6198ce41a0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:25.794923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e9b57-f470-4ba9-9ee3-178c57344d05 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:25.803691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccaf74-6cad-4b03-9412-19bb30c874d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:25.804088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccaf74-6cad-4b03-9412-19bb30c874d0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.804358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccaf74-6cad-4b03-9412-19bb30c874d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.804631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccaf74-6cad-4b03-9412-19bb30c874d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.827705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:25.892991 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccaf74-6cad-4b03-9412-19bb30c874d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:25.894515 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccaf74-6cad-4b03-9412-19bb30c874d0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:25.930266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e9b57-f470-4ba9-9ee3-178c57344d05 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:25.934133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:25.982935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d96d9-7d4e-4fad-9bce-8f1d57e01900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:25.983579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d96d9-7d4e-4fad-9bce-8f1d57e01900 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.983856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d96d9-7d4e-4fad-9bce-8f1d57e01900 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.984127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d96d9-7d4e-4fad-9bce-8f1d57e01900 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.998244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d96d9-7d4e-4fad-9bce-8f1d57e01900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-428d96d9-7d4e-4fad-9bce-8f1d57e01900 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:26.006235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:26.007293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:26.008489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 114/222] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:26.009362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d96d9-7d4e-4fad-9bce-8f1d57e01900 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:26.024431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 109/223] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:26.025287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccaf74-6cad-4b03-9412-19bb30c874d0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:26.121511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:26.135739 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccaf74-6cad-4b03-9412-19bb30c874d0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:26.150118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d96d9-7d4e-4fad-9bce-8f1d57e01900 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:26.165154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:26.173684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23bbc7-588f-430c-81fe-eb0c8892957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:26.174302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23bbc7-588f-430c-81fe-eb0c8892957f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:26.174570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23bbc7-588f-430c-81fe-eb0c8892957f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:26.174839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23bbc7-588f-430c-81fe-eb0c8892957f None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:26.175507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23bbc7-588f-430c-81fe-eb0c8892957f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:26.176311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23bbc7-588f-430c-81fe-eb0c8892957f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:26.178164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23bbc7-588f-430c-81fe-eb0c8892957f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:26.186997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 110/224] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:26.187815 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23bbc7-588f-430c-81fe-eb0c8892957f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:26.198344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1f11c-80eb-4aad-b0a5-84aad98945d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:26.200727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1f11c-80eb-4aad-b0a5-84aad98945d1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:26.201636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1f11c-80eb-4aad-b0a5-84aad98945d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:26.202061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1f11c-80eb-4aad-b0a5-84aad98945d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:26.221501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1f11c-80eb-4aad-b0a5-84aad98945d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-34f1f11c-80eb-4aad-b0a5-84aad98945d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:26.226966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:26.228182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:26.229186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:26.235061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 115/225] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:26.235999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1f11c-80eb-4aad-b0a5-84aad98945d1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:26.324610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23bbc7-588f-430c-81fe-eb0c8892957f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:26.349643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1f11c-80eb-4aad-b0a5-84aad98945d1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:26.371071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1447a9-a95c-4d04-aadb-609704d01bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:26.372590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1447a9-a95c-4d04-aadb-609704d01bcc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:26.373015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1447a9-a95c-4d04-aadb-609704d01bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:26.373740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1447a9-a95c-4d04-aadb-609704d01bcc None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:26.375269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1447a9-a95c-4d04-aadb-609704d01bcc None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:26.377616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1447a9-a95c-4d04-aadb-609704d01bcc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:26.380797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 111/226] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:26 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:26.382040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1447a9-a95c-4d04-aadb-609704d01bcc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:26.389266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75456b34-6138-4ae8-9779-10ca5a8b65c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:26.390058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75456b34-6138-4ae8-9779-10ca5a8b65c4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:26.390432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75456b34-6138-4ae8-9779-10ca5a8b65c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:26.390791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75456b34-6138-4ae8-9779-10ca5a8b65c4 None admin] PATH_INFO: `/v3/users/33f498cc19a44628933942cd4aaad2d4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:26.392037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75456b34-6138-4ae8-9779-10ca5a8b65c4 None admin] RBAC: Authorizing `identity:get_user(user_id=33f498cc19a44628933942cd4aaad2d4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:26.393605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75456b34-6138-4ae8-9779-10ca5a8b65c4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:26.395979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 116/227] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:26 2022] GET /identity/v3/users/33f498cc19a44628933942cd4aaad2d4 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:26.396873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75456b34-6138-4ae8-9779-10ca5a8b65c4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:26.408060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:26.461050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:26.514227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75456b34-6138-4ae8-9779-10ca5a8b65c4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:26.521335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1447a9-a95c-4d04-aadb-609704d01bcc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:26.551073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e373734-5bf6-4953-b13f-036548b1fb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:26.552074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e373734-5bf6-4953-b13f-036548b1fb79 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:26.552495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e373734-5bf6-4953-b13f-036548b1fb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:26.552867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e373734-5bf6-4953-b13f-036548b1fb79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:26.553756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e373734-5bf6-4953-b13f-036548b1fb79 None admin] RBAC: Adding query filter params (role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, scope.project.id=d61e52cca9584059bf484cabd52d2808, user.id=f461d4cb769f4b658b434a594e78c32c) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:26.554163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e373734-5bf6-4953-b13f-036548b1fb79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:26.555684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e373734-5bf6-4953-b13f-036548b1fb79 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:26.564384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 117/228] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:26 2022] GET /identity/v3/role_assignments?user.id=f461d4cb769f4b658b434a594e78c32c&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:26.565342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e373734-5bf6-4953-b13f-036548b1fb79 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:26.568775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfccd4-7b83-4d53-848f-99bf94b1cab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:26.569549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfccd4-7b83-4d53-848f-99bf94b1cab2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:26.569924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfccd4-7b83-4d53-848f-99bf94b1cab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:26.570315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfccd4-7b83-4d53-848f-99bf94b1cab2 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:26.571468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfccd4-7b83-4d53-848f-99bf94b1cab2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:26.571468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfccd4-7b83-4d53-848f-99bf94b1cab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:26.572986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfccd4-7b83-4d53-848f-99bf94b1cab2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:26.583070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 112/229] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:26.589079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfccd4-7b83-4d53-848f-99bf94b1cab2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:26.655617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:26.694835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e373734-5bf6-4953-b13f-036548b1fb79 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:26.737041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86109f25-6949-4591-9194-b66b3b904014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:26.738490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86109f25-6949-4591-9194-b66b3b904014 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:26.739049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86109f25-6949-4591-9194-b66b3b904014 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:26.739608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86109f25-6949-4591-9194-b66b3b904014 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:26.741020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86109f25-6949-4591-9194-b66b3b904014 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:26.742782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86109f25-6949-4591-9194-b66b3b904014 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:26.745553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 118/230] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:26 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:26.751375 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86109f25-6949-4591-9194-b66b3b904014 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:26.836521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfccd4-7b83-4d53-848f-99bf94b1cab2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:26.859855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:26.882455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86109f25-6949-4591-9194-b66b3b904014 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:26.892015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc49c3-17b7-44b0-901f-75abdb4534f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:26.892719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc49c3-17b7-44b0-901f-75abdb4534f9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:26.893006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc49c3-17b7-44b0-901f-75abdb4534f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:26.893283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc49c3-17b7-44b0-901f-75abdb4534f9 None admin] PATH_INFO: `/v3/users/33f498cc19a44628933942cd4aaad2d4` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:26.898259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc49c3-17b7-44b0-901f-75abdb4534f9 None admin] RBAC: Authorizing `identity:get_user(user_id=33f498cc19a44628933942cd4aaad2d4)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:26.899863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc49c3-17b7-44b0-901f-75abdb4534f9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:26.901741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 113/231] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:26 2022] GET /identity/v3/users/33f498cc19a44628933942cd4aaad2d4 => generated 283 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:26.909063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc49c3-17b7-44b0-901f-75abdb4534f9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:26.925725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a3876-51ab-410d-805d-7c5e17c70928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:26.926774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3876-51ab-410d-805d-7c5e17c70928 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:26.927275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3876-51ab-410d-805d-7c5e17c70928 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:26.927744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3876-51ab-410d-805d-7c5e17c70928 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808/users/33f498cc19a44628933942cd4aaad2d4/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:26.929967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a3876-51ab-410d-805d-7c5e17c70928 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61e52cca9584059bf484cabd52d2808, user_id=33f498cc19a44628933942cd4aaad2d4, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:26.932150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a3876-51ab-410d-805d-7c5e17c70928 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:27.025023 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:27.040299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 119/232] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:26 2022] PUT /identity/v3/projects/d61e52cca9584059bf484cabd52d2808/users/33f498cc19a44628933942cd4aaad2d4/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:27.042759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc49c3-17b7-44b0-901f-75abdb4534f9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:27.085927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cd5b3-97a5-40e4-b33e-dd86c8ded218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:27.086590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cd5b3-97a5-40e4-b33e-dd86c8ded218 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:27.086874 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cd5b3-97a5-40e4-b33e-dd86c8ded218 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:27.087192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cd5b3-97a5-40e4-b33e-dd86c8ded218 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:27.088420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cd5b3-97a5-40e4-b33e-dd86c8ded218 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:27.090078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cd5b3-97a5-40e4-b33e-dd86c8ded218 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:27.092194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 114/233] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:26 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:27.097977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a3876-51ab-410d-805d-7c5e17c70928 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:27.236687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a3876-51ab-410d-805d-7c5e17c70928 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:27.272638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462c755-df7f-427e-b6fe-8db89d508bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:27.273308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462c755-df7f-427e-b6fe-8db89d508bc7 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:27.273614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462c755-df7f-427e-b6fe-8db89d508bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:27.273885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462c755-df7f-427e-b6fe-8db89d508bc7 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808/users/33f498cc19a44628933942cd4aaad2d4/roles/acfe3eeb91964f62abb1d8a84c36dccc` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:27.276075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462c755-df7f-427e-b6fe-8db89d508bc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61e52cca9584059bf484cabd52d2808, user_id=33f498cc19a44628933942cd4aaad2d4, role_id=acfe3eeb91964f62abb1d8a84c36dccc)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:27.277532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462c755-df7f-427e-b6fe-8db89d508bc7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:27.383648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 120/234] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:27 2022] PUT /identity/v3/projects/d61e52cca9584059bf484cabd52d2808/users/33f498cc19a44628933942cd4aaad2d4/roles/acfe3eeb91964f62abb1d8a84c36dccc => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:27.842151 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e695ede-bc7f-4871-a238-cfb7f9ed6ecd None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:27.842661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e695ede-bc7f-4871-a238-cfb7f9ed6ecd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:27.842736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e695ede-bc7f-4871-a238-cfb7f9ed6ecd None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:27.844396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 115/235] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:27.849916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa71ad-b95b-4822-97e8-cb40d661f0e1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:27.850320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa71ad-b95b-4822-97e8-cb40d661f0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:27.850508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa71ad-b95b-4822-97e8-cb40d661f0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:27.871637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89feb8-9195-400e-ae7b-dd79c982f043 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:27.871949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89feb8-9195-400e-ae7b-dd79c982f043 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:27.872190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89feb8-9195-400e-ae7b-dd79c982f043 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:27.873249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f0fa71ad-b95b-4822-97e8-cb40d661f0e1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:27.873798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 116/236] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:27.874529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f0fa71ad-b95b-4822-97e8-cb40d661f0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:27.883682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718a8a-f4e2-4b79-896e-a9df585cbdf6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:27.883954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718a8a-f4e2-4b79-896e-a9df585cbdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:27.884231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718a8a-f4e2-4b79-896e-a9df585cbdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:27.926623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-01718a8a-f4e2-4b79-896e-a9df585cbdf6 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:27.927852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-01718a8a-f4e2-4b79-896e-a9df585cbdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:27.945842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:27.970015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 121/237] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:28.013884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:28.040194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 117/238] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:28.340615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa71ad-b95b-4822-97e8-cb40d661f0e1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:28.362028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718a8a-f4e2-4b79-896e-a9df585cbdf6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:28.477544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718a8a-f4e2-4b79-896e-a9df585cbdf6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:28.516757 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737f223-f035-46ae-a5e2-5ef0d6c39dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:28.517391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737f223-f035-46ae-a5e2-5ef0d6c39dda None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:28.517689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737f223-f035-46ae-a5e2-5ef0d6c39dda None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:28.518951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737f223-f035-46ae-a5e2-5ef0d6c39dda None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:28.538599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa71ad-b95b-4822-97e8-cb40d661f0e1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:28.541325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737f223-f035-46ae-a5e2-5ef0d6c39dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-f737f223-f035-46ae-a5e2-5ef0d6c39dda None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:28.546095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:28.546981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:28.547776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 118/239] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:28.559790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737f223-f035-46ae-a5e2-5ef0d6c39dda None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:28.603279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382b47f-b37e-408e-99cd-15779dba4dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:28.603834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382b47f-b37e-408e-99cd-15779dba4dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:28.604104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382b47f-b37e-408e-99cd-15779dba4dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:28.604369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382b47f-b37e-408e-99cd-15779dba4dc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:28.606835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382b47f-b37e-408e-99cd-15779dba4dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:28.608272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382b47f-b37e-408e-99cd-15779dba4dc4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:28.611034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 122/240] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:28.615847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382b47f-b37e-408e-99cd-15779dba4dc4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:28.655743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:28.729536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:28.765918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737f223-f035-46ae-a5e2-5ef0d6c39dda None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:28.783934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382b47f-b37e-408e-99cd-15779dba4dc4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:28.820908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab5f58-4cdd-4f92-8c17-61599ff86883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:28.821555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5f58-4cdd-4f92-8c17-61599ff86883 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:28.821832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5f58-4cdd-4f92-8c17-61599ff86883 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:28.822103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5f58-4cdd-4f92-8c17-61599ff86883 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:28.822766 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab5f58-4cdd-4f92-8c17-61599ff86883 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:28.824194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab5f58-4cdd-4f92-8c17-61599ff86883 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:28.839206 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e931b-e32e-4146-bccc-8611710f547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:28.839862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e931b-e32e-4146-bccc-8611710f547b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:28.840138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e931b-e32e-4146-bccc-8611710f547b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:28.840404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e931b-e32e-4146-bccc-8611710f547b None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:28.841120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e931b-e32e-4146-bccc-8611710f547b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:28.841423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e931b-e32e-4146-bccc-8611710f547b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:28.842952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e931b-e32e-4146-bccc-8611710f547b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:28.852864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 119/241] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:28.858423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e931b-e32e-4146-bccc-8611710f547b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:28.903218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 123/242] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:28 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:12:29.002915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e931b-e32e-4146-bccc-8611710f547b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:29.017759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:29.045688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b3501-3dae-4412-9ad6-ca324a79510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:29.046616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b3501-3dae-4412-9ad6-ca324a79510a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.047052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b3501-3dae-4412-9ad6-ca324a79510a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.047462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b3501-3dae-4412-9ad6-ca324a79510a None admin] PATH_INFO: `/v3/users/93febba9030a47de93dc55826ef62a3d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.068159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b3501-3dae-4412-9ad6-ca324a79510a None admin] RBAC: Authorizing `identity:get_user(user_id=93febba9030a47de93dc55826ef62a3d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:29.070024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b3501-3dae-4412-9ad6-ca324a79510a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:29.071993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 120/243] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:28 2022] GET /identity/v3/users/93febba9030a47de93dc55826ef62a3d => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:29.078338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab5f58-4cdd-4f92-8c17-61599ff86883 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:29.167178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ec488-bde1-4b4e-bb06-8d0356711506 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.167774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ec488-bde1-4b4e-bb06-8d0356711506 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.168414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ec488-bde1-4b4e-bb06-8d0356711506 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.171875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 121/244] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:29.177825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd814dc-3b72-4bf5-8c96-d2aae284ef72 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.184471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd814dc-3b72-4bf5-8c96-d2aae284ef72 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.184471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd814dc-3b72-4bf5-8c96-d2aae284ef72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.199960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1fd814dc-3b72-4bf5-8c96-d2aae284ef72 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:29.201565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1fd814dc-3b72-4bf5-8c96-d2aae284ef72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:29.211907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab5f58-4cdd-4f92-8c17-61599ff86883 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:29.228933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:29.253917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21fd39-4ad7-4bc5-85cf-acecb5df24e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:29.255481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21fd39-4ad7-4bc5-85cf-acecb5df24e7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.255481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21fd39-4ad7-4bc5-85cf-acecb5df24e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.255481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21fd39-4ad7-4bc5-85cf-acecb5df24e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.311175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 122/245] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:29.340762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21fd39-4ad7-4bc5-85cf-acecb5df24e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:29.342604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21fd39-4ad7-4bc5-85cf-acecb5df24e7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:29.462659 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 124/246] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:29.470574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd814dc-3b72-4bf5-8c96-d2aae284ef72 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:29.547134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3213bd-09bb-43d1-a1b2-d3b185a3d047 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.547761 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3213bd-09bb-43d1-a1b2-d3b185a3d047 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.548282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3213bd-09bb-43d1-a1b2-d3b185a3d047 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.549885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 125/247] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:12:29.561292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c9775-2748-4fb9-be90-c3a67a5047cc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.561967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c9775-2748-4fb9-be90-c3a67a5047cc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.562488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c9775-2748-4fb9-be90-c3a67a5047cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.584296 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b59c9775-2748-4fb9-be90-c3a67a5047cc None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:29.585850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b59c9775-2748-4fb9-be90-c3a67a5047cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:29.589613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd814dc-3b72-4bf5-8c96-d2aae284ef72 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:29.611596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:29.613082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:29.633368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b56387-fd9d-4e8d-95f6-4b3022ab1d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:29.633793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b56387-fd9d-4e8d-95f6-4b3022ab1d42 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.634071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b56387-fd9d-4e8d-95f6-4b3022ab1d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.634343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b56387-fd9d-4e8d-95f6-4b3022ab1d42 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.637053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b56387-fd9d-4e8d-95f6-4b3022ab1d42 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:29.638018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b56387-fd9d-4e8d-95f6-4b3022ab1d42 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:29.639847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 123/248] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:29 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:29.640806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b56387-fd9d-4e8d-95f6-4b3022ab1d42 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:29.689414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 126/249] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:29.690319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c9775-2748-4fb9-be90-c3a67a5047cc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:29.803459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b56387-fd9d-4e8d-95f6-4b3022ab1d42 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:29.855495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c9775-2748-4fb9-be90-c3a67a5047cc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:29.883973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bcf1c-9cd1-496c-ad68-57b342ccd57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:29.884651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bcf1c-9cd1-496c-ad68-57b342ccd57c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.884894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bcf1c-9cd1-496c-ad68-57b342ccd57c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.885167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bcf1c-9cd1-496c-ad68-57b342ccd57c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.911503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bcf1c-9cd1-496c-ad68-57b342ccd57c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-fe6bcf1c-9cd1-496c-ad68-57b342ccd57c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:29.917057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:29.917975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:29.918767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 124/250] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:29.919433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89af84-344d-44b3-a516-dbefcec34afe None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.919987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89af84-344d-44b3-a516-dbefcec34afe None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.919987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89af84-344d-44b3-a516-dbefcec34afe None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.926788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 125/251] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:29.930728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2039b-66af-452c-821a-96b9cd57e62c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.931008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2039b-66af-452c-821a-96b9cd57e62c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.931284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2039b-66af-452c-821a-96b9cd57e62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.937243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87796-854b-40b6-a403-5a22b4f15100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:29.937874 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87796-854b-40b6-a403-5a22b4f15100 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.938161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87796-854b-40b6-a403-5a22b4f15100 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.938435 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87796-854b-40b6-a403-5a22b4f15100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.939176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b87796-854b-40b6-a403-5a22b4f15100 None admin] RBAC: Adding query filter params (role.id=d529d6d058424562a72162b0c257851f, scope.project.id=d61e52cca9584059bf484cabd52d2808, user.id=93febba9030a47de93dc55826ef62a3d) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:29.939484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b87796-854b-40b6-a403-5a22b4f15100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:29.941267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b87796-854b-40b6-a403-5a22b4f15100 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:29.957526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 127/252] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:29 2022] GET /identity/v3/role_assignments?user.id=93febba9030a47de93dc55826ef62a3d&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=d529d6d058424562a72162b0c257851f => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:29.958888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87796-854b-40b6-a403-5a22b4f15100 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:29.970939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-69d2039b-66af-452c-821a-96b9cd57e62c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:29.974671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-69d2039b-66af-452c-821a-96b9cd57e62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:29.979733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:30.017296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:30.112499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 126/253] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:30.117489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2039b-66af-452c-821a-96b9cd57e62c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:30.134170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87796-854b-40b6-a403-5a22b4f15100 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:30.174290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4d5f0-5797-42b7-ad78-cf0a4aca437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:30.174912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4d5f0-5797-42b7-ad78-cf0a4aca437a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:30.175193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4d5f0-5797-42b7-ad78-cf0a4aca437a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:30.175471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4d5f0-5797-42b7-ad78-cf0a4aca437a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:30.176837 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4d5f0-5797-42b7-ad78-cf0a4aca437a None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:30.177171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4d5f0-5797-42b7-ad78-cf0a4aca437a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:30.178749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4d5f0-5797-42b7-ad78-cf0a4aca437a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:30.190393 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 128/254] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:30.197976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4d5f0-5797-42b7-ad78-cf0a4aca437a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:30.319119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2039b-66af-452c-821a-96b9cd57e62c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:30.333809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:30.381674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f92c09-3dd5-44d9-99f6-1bbcceeda229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:30.382449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92c09-3dd5-44d9-99f6-1bbcceeda229 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:30.382774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92c09-3dd5-44d9-99f6-1bbcceeda229 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:30.383149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92c09-3dd5-44d9-99f6-1bbcceeda229 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:30.394657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4d5f0-5797-42b7-ad78-cf0a4aca437a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:30.408209 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f92c09-3dd5-44d9-99f6-1bbcceeda229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-25f92c09-3dd5-44d9-99f6-1bbcceeda229 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:30.414093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:30.414968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:30.415776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 127/255] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:30.426785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f92c09-3dd5-44d9-99f6-1bbcceeda229 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:30.453168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:30.479397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875d2d9-9a78-45e3-8649-221ceac82b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:30.480302 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875d2d9-9a78-45e3-8649-221ceac82b16 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:30.480736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875d2d9-9a78-45e3-8649-221ceac82b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:30.481297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875d2d9-9a78-45e3-8649-221ceac82b16 None admin] PATH_INFO: `/v3/users/f461d4cb769f4b658b434a594e78c32c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:30.483957 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875d2d9-9a78-45e3-8649-221ceac82b16 None admin] RBAC: Authorizing `identity:get_user(user_id=f461d4cb769f4b658b434a594e78c32c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:30.485408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875d2d9-9a78-45e3-8649-221ceac82b16 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:30.487198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 129/256] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:30 2022] GET /identity/v3/users/f461d4cb769f4b658b434a594e78c32c => generated 297 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:30.489495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875d2d9-9a78-45e3-8649-221ceac82b16 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:30.609898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f92c09-3dd5-44d9-99f6-1bbcceeda229 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:30.643352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875d2d9-9a78-45e3-8649-221ceac82b16 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:30.685567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54c9ea-949d-4c9a-bf3a-cac987813fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:30.686199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54c9ea-949d-4c9a-bf3a-cac987813fdc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:30.686468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54c9ea-949d-4c9a-bf3a-cac987813fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:30.686737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54c9ea-949d-4c9a-bf3a-cac987813fdc None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:30.687405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54c9ea-949d-4c9a-bf3a-cac987813fdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:30.687813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54c9ea-949d-4c9a-bf3a-cac987813fdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:30.693592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54c9ea-949d-4c9a-bf3a-cac987813fdc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:30.694561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810bab3-ae70-4877-b33c-4d97ba84d892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:30.695204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810bab3-ae70-4877-b33c-4d97ba84d892 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:30.695487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810bab3-ae70-4877-b33c-4d97ba84d892 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:30.695765 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810bab3-ae70-4877-b33c-4d97ba84d892 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:30.708447 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 128/257] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:30.709747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54c9ea-949d-4c9a-bf3a-cac987813fdc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:30.711039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:30.718719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810bab3-ae70-4877-b33c-4d97ba84d892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a810bab3-ae70-4877-b33c-4d97ba84d892 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:30.730464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:30.731338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:30.732136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:30.737774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 130/258] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:30.739192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810bab3-ae70-4877-b33c-4d97ba84d892 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:30.839605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:30.865978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54c9ea-949d-4c9a-bf3a-cac987813fdc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:30.913098 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a387f9-2361-460c-9bec-bc1a2c1cd056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:30.914764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a387f9-2361-460c-9bec-bc1a2c1cd056 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:30.915103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a387f9-2361-460c-9bec-bc1a2c1cd056 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:30.915385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a387f9-2361-460c-9bec-bc1a2c1cd056 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:30.919603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a387f9-2361-460c-9bec-bc1a2c1cd056 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:30.921129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a387f9-2361-460c-9bec-bc1a2c1cd056 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:30.923195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 129/259] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:30 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:30.924183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a387f9-2361-460c-9bec-bc1a2c1cd056 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:30.958980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810bab3-ae70-4877-b33c-4d97ba84d892 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:31.011025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bc26e-08ef-45c3-9826-4869924bd9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:31.011906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bc26e-08ef-45c3-9826-4869924bd9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:31.012334 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bc26e-08ef-45c3-9826-4869924bd9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:31.012750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bc26e-08ef-45c3-9826-4869924bd9f8 None admin] PATH_INFO: `/v3/users/33f498cc19a44628933942cd4aaad2d4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:31.014058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bc26e-08ef-45c3-9826-4869924bd9f8 None admin] RBAC: Authorizing `identity:get_user(user_id=33f498cc19a44628933942cd4aaad2d4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:31.015679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bc26e-08ef-45c3-9826-4869924bd9f8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:31.017411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 131/260] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:30 2022] GET /identity/v3/users/33f498cc19a44628933942cd4aaad2d4 => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:31.018388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bc26e-08ef-45c3-9826-4869924bd9f8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:31.091301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:31.128074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bc26e-08ef-45c3-9826-4869924bd9f8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:31.156206 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a387f9-2361-460c-9bec-bc1a2c1cd056 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:31.171189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83d3e9-968a-4e2c-9cb2-cef669e7028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:31.172104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83d3e9-968a-4e2c-9cb2-cef669e7028b None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:31.172565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83d3e9-968a-4e2c-9cb2-cef669e7028b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:31.173127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83d3e9-968a-4e2c-9cb2-cef669e7028b None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808/users/f461d4cb769f4b658b434a594e78c32c/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:31.177319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83d3e9-968a-4e2c-9cb2-cef669e7028b None admin] RBAC: Authorizing `identity:create_grant(project_id=d61e52cca9584059bf484cabd52d2808, user_id=f461d4cb769f4b658b434a594e78c32c, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:31.179056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83d3e9-968a-4e2c-9cb2-cef669e7028b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:31.206910 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ee778-d352-441d-97ed-21a4fdc6dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:31.207868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ee778-d352-441d-97ed-21a4fdc6dbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:31.208380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ee778-d352-441d-97ed-21a4fdc6dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:31.209179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ee778-d352-441d-97ed-21a4fdc6dbf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:31.210142 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ee778-d352-441d-97ed-21a4fdc6dbf3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:31.210713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ee778-d352-441d-97ed-21a4fdc6dbf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:31.212422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ee778-d352-441d-97ed-21a4fdc6dbf3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:31.223636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 130/261] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:31.229852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ee778-d352-441d-97ed-21a4fdc6dbf3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:31.239203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:31.279292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 132/262] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:31 2022] PUT /identity/v3/projects/d61e52cca9584059bf484cabd52d2808/users/f461d4cb769f4b658b434a594e78c32c/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:31.280470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83d3e9-968a-4e2c-9cb2-cef669e7028b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:31.369949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ee778-d352-441d-97ed-21a4fdc6dbf3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:31.410981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9c3e6-4ac7-408e-8d41-7766cf10c4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:31.411927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9c3e6-4ac7-408e-8d41-7766cf10c4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:31.412426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9c3e6-4ac7-408e-8d41-7766cf10c4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:31.412937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9c3e6-4ac7-408e-8d41-7766cf10c4d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:31.496721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9c3e6-4ac7-408e-8d41-7766cf10c4d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:31.498614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9c3e6-4ac7-408e-8d41-7766cf10c4d6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:31.503926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83d3e9-968a-4e2c-9cb2-cef669e7028b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:31.509075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:31.565171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:31.594373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268286b0-ccf1-4116-99c3-47e497eed811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:31.595027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268286b0-ccf1-4116-99c3-47e497eed811 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:31.595310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268286b0-ccf1-4116-99c3-47e497eed811 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:31.595579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268286b0-ccf1-4116-99c3-47e497eed811 None admin] PATH_INFO: `/v3/users/33f498cc19a44628933942cd4aaad2d4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:31.596762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268286b0-ccf1-4116-99c3-47e497eed811 None admin] RBAC: Authorizing `identity:get_user(user_id=33f498cc19a44628933942cd4aaad2d4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:31.598788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268286b0-ccf1-4116-99c3-47e497eed811 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:31.604525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 133/263] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:31 2022] GET /identity/v3/users/33f498cc19a44628933942cd4aaad2d4 => generated 283 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:31.622201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268286b0-ccf1-4116-99c3-47e497eed811 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:31.623608 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 131/264] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:31.630149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9c3e6-4ac7-408e-8d41-7766cf10c4d6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:31.825716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268286b0-ccf1-4116-99c3-47e497eed811 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:31.835383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9c3e6-4ac7-408e-8d41-7766cf10c4d6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:31.853814 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:31.892927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc02d48-e8e4-490f-b87c-de7eaab809c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:31.893597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc02d48-e8e4-490f-b87c-de7eaab809c4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:31.893878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc02d48-e8e4-490f-b87c-de7eaab809c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:31.894409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc02d48-e8e4-490f-b87c-de7eaab809c4 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:31.896016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc02d48-e8e4-490f-b87c-de7eaab809c4 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:31.897505 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc02d48-e8e4-490f-b87c-de7eaab809c4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:31.899489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 132/265] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:31 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:31.905258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc02d48-e8e4-490f-b87c-de7eaab809c4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:31.918170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd3df5-c635-4f62-b573-17af66665a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:31.918922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd3df5-c635-4f62-b573-17af66665a72 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:31.919217 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd3df5-c635-4f62-b573-17af66665a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:31.919500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd3df5-c635-4f62-b573-17af66665a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.026700 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc02d48-e8e4-490f-b87c-de7eaab809c4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:32.049836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:32.054382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd3df5-c635-4f62-b573-17af66665a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:32.055888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd3df5-c635-4f62-b573-17af66665a72 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:32.118192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfed800-3afe-44f2-9a5d-2ffce513640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:32.118958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfed800-3afe-44f2-9a5d-2ffce513640b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.119336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfed800-3afe-44f2-9a5d-2ffce513640b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.119671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfed800-3afe-44f2-9a5d-2ffce513640b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.120481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfed800-3afe-44f2-9a5d-2ffce513640b None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=d61e52cca9584059bf484cabd52d2808, user.id=33f498cc19a44628933942cd4aaad2d4) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:32.120854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfed800-3afe-44f2-9a5d-2ffce513640b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:32.122430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfed800-3afe-44f2-9a5d-2ffce513640b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:32.138277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 133/266] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:31 2022] GET /identity/v3/role_assignments?user.id=33f498cc19a44628933942cd4aaad2d4&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:32.140078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcfb06-ff88-426d-916f-d811ff56fd2f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.140535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcfb06-ff88-426d-916f-d811ff56fd2f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.141329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcfb06-ff88-426d-916f-d811ff56fd2f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.148391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 134/267] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:32.154179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c02d59-6966-438d-8d79-c434446dd642 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.154743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c02d59-6966-438d-8d79-c434446dd642 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.155178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c02d59-6966-438d-8d79-c434446dd642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.193389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e1c02d59-6966-438d-8d79-c434446dd642 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:32.195117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e1c02d59-6966-438d-8d79-c434446dd642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:32.234267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:32.239340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 134/268] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:32.241084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dccff-f46d-4e55-998f-2badd5dc511d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.241619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dccff-f46d-4e55-998f-2badd5dc511d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.242206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dccff-f46d-4e55-998f-2badd5dc511d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.244128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 135/269] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:32.245179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9dccff-f46d-4e55-998f-2badd5dc511d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:32.299527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 135/270] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:32.300789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5114b8e-7fcb-44b3-86b7-74bcc11b71a5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.301079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5114b8e-7fcb-44b3-86b7-74bcc11b71a5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.301350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5114b8e-7fcb-44b3-86b7-74bcc11b71a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.351783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b5114b8e-7fcb-44b3-86b7-74bcc11b71a5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:32.352989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b5114b8e-7fcb-44b3-86b7-74bcc11b71a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:32.425358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 136/271] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:32.444425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:32.460047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9dccff-f46d-4e55-998f-2badd5dc511d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:32.499353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774a587-b0f0-4200-ac28-9df283df81f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:32.499995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774a587-b0f0-4200-ac28-9df283df81f1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.500278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774a587-b0f0-4200-ac28-9df283df81f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.500549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774a587-b0f0-4200-ac28-9df283df81f1 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.501767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774a587-b0f0-4200-ac28-9df283df81f1 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:32.503292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774a587-b0f0-4200-ac28-9df283df81f1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:32.505786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 136/272] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:32 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:32.511821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5114b8e-7fcb-44b3-86b7-74bcc11b71a5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:32.639432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:32.649036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5114b8e-7fcb-44b3-86b7-74bcc11b71a5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:32.679196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b366-636c-4ea1-90ee-600d0f8419df None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.680091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b366-636c-4ea1-90ee-600d0f8419df None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.680598 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b366-636c-4ea1-90ee-600d0f8419df None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.682245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 137/273] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:32.687823 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222638b6-4a5b-4544-80f4-fdd9409121c9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.688094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222638b6-4a5b-4544-80f4-fdd9409121c9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.688368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222638b6-4a5b-4544-80f4-fdd9409121c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.688561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c80ac1-359f-4719-a508-4f8e135b5165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:32.700775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c80ac1-359f-4719-a508-4f8e135b5165 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.700775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c80ac1-359f-4719-a508-4f8e135b5165 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.700775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c80ac1-359f-4719-a508-4f8e135b5165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.700775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c80ac1-359f-4719-a508-4f8e135b5165 None admin] RBAC: Adding query filter params (role.id=acfe3eeb91964f62abb1d8a84c36dccc, scope.project.id=d61e52cca9584059bf484cabd52d2808, user.id=33f498cc19a44628933942cd4aaad2d4) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:32.700775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c80ac1-359f-4719-a508-4f8e135b5165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:32.700775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c80ac1-359f-4719-a508-4f8e135b5165 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:32.717398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 137/274] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:32 2022] GET /identity/v3/role_assignments?user.id=33f498cc19a44628933942cd4aaad2d4&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=acfe3eeb91964f62abb1d8a84c36dccc => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:32.718422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-222638b6-4a5b-4544-80f4-fdd9409121c9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:32.719664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-222638b6-4a5b-4544-80f4-fdd9409121c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:32.798496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c80ac1-359f-4719-a508-4f8e135b5165 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:32.857220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 138/275] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 05:12:32.858170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222638b6-4a5b-4544-80f4-fdd9409121c9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:32.976435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c80ac1-359f-4719-a508-4f8e135b5165 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.027521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:33.031458 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba401f7a-dd3b-454c-9b31-b2c90d26ca7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.032142 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba401f7a-dd3b-454c-9b31-b2c90d26ca7f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.032607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba401f7a-dd3b-454c-9b31-b2c90d26ca7f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.032886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba401f7a-dd3b-454c-9b31-b2c90d26ca7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.034977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba401f7a-dd3b-454c-9b31-b2c90d26ca7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.036566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba401f7a-dd3b-454c-9b31-b2c90d26ca7f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.038483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 138/276] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:33.045560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba401f7a-dd3b-454c-9b31-b2c90d26ca7f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.047301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222638b6-4a5b-4544-80f4-fdd9409121c9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.092256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:33.101711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedee5fb-7c74-4ebd-ae7c-cde010e7aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.102406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedee5fb-7c74-4ebd-ae7c-cde010e7aa7c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.102691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedee5fb-7c74-4ebd-ae7c-cde010e7aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.102964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedee5fb-7c74-4ebd-ae7c-cde010e7aa7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.117302 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedee5fb-7c74-4ebd-ae7c-cde010e7aa7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-bedee5fb-7c74-4ebd-ae7c-cde010e7aa7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:33.122649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:33.124657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:33.126966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 139/277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:33.133484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedee5fb-7c74-4ebd-ae7c-cde010e7aa7c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.198341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba401f7a-dd3b-454c-9b31-b2c90d26ca7f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.245177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de409b8-238a-4637-b98d-c82e26d03066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.245736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de409b8-238a-4637-b98d-c82e26d03066 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.246013 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de409b8-238a-4637-b98d-c82e26d03066 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.246283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de409b8-238a-4637-b98d-c82e26d03066 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.247425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de409b8-238a-4637-b98d-c82e26d03066 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.249371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de409b8-238a-4637-b98d-c82e26d03066 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.276316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedee5fb-7c74-4ebd-ae7c-cde010e7aa7c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.313156 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79120a7e-2f70-47ff-a70b-25671daffd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.314280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79120a7e-2f70-47ff-a70b-25671daffd2c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.314280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79120a7e-2f70-47ff-a70b-25671daffd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.314512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79120a7e-2f70-47ff-a70b-25671daffd2c None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.314988 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79120a7e-2f70-47ff-a70b-25671daffd2c None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:33.315289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79120a7e-2f70-47ff-a70b-25671daffd2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.317204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79120a7e-2f70-47ff-a70b-25671daffd2c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.328343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 140/278] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:33.347574 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79120a7e-2f70-47ff-a70b-25671daffd2c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.348240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:33.381495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 139/279] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:33 2022] POST /identity/v3/users => generated 293 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:12:33.381495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de409b8-238a-4637-b98d-c82e26d03066 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.482504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79120a7e-2f70-47ff-a70b-25671daffd2c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.509762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:33.532423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de409b8-238a-4637-b98d-c82e26d03066 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.533858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18e48d-e8af-45fe-bffd-b4dd41de004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.534608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18e48d-e8af-45fe-bffd-b4dd41de004d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.535114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18e48d-e8af-45fe-bffd-b4dd41de004d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.535421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18e48d-e8af-45fe-bffd-b4dd41de004d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.548362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18e48d-e8af-45fe-bffd-b4dd41de004d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ff18e48d-e8af-45fe-bffd-b4dd41de004d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:33.553816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:33.555635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:33.556440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 141/280] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:33.561172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18e48d-e8af-45fe-bffd-b4dd41de004d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.582799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecf041-2bb8-40e0-9711-87f82817d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.583682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecf041-2bb8-40e0-9711-87f82817d967 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.584249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecf041-2bb8-40e0-9711-87f82817d967 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.584678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecf041-2bb8-40e0-9711-87f82817d967 None admin] PATH_INFO: `/v3/users/ab5c16b8f13c42e2b687998cfb3a66f7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.601573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecf041-2bb8-40e0-9711-87f82817d967 None admin] RBAC: Authorizing `identity:get_user(user_id=ab5c16b8f13c42e2b687998cfb3a66f7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.603056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecf041-2bb8-40e0-9711-87f82817d967 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.604892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 140/281] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/users/ab5c16b8f13c42e2b687998cfb3a66f7 => generated 293 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:33.611249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecf041-2bb8-40e0-9711-87f82817d967 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.657273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:33.697110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18e48d-e8af-45fe-bffd-b4dd41de004d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.744735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1552e12-f4c3-40dc-b3db-f912d9ca0955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.745358 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1552e12-f4c3-40dc-b3db-f912d9ca0955 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.745671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1552e12-f4c3-40dc-b3db-f912d9ca0955 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.745943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1552e12-f4c3-40dc-b3db-f912d9ca0955 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.746620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1552e12-f4c3-40dc-b3db-f912d9ca0955 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:33.746924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1552e12-f4c3-40dc-b3db-f912d9ca0955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.748319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1552e12-f4c3-40dc-b3db-f912d9ca0955 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.752646 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecf041-2bb8-40e0-9711-87f82817d967 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.758362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 142/282] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:33.765607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1552e12-f4c3-40dc-b3db-f912d9ca0955 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.801383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94369e-b50d-42fc-8e7e-e1e61b94ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.802082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94369e-b50d-42fc-8e7e-e1e61b94ed88 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.802374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94369e-b50d-42fc-8e7e-e1e61b94ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.802658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94369e-b50d-42fc-8e7e-e1e61b94ed88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.803414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94369e-b50d-42fc-8e7e-e1e61b94ed88 None admin] RBAC: Adding query filter params (role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, user.id=ab5c16b8f13c42e2b687998cfb3a66f7, scope.system=all) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:33.803726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94369e-b50d-42fc-8e7e-e1e61b94ed88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.805207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94369e-b50d-42fc-8e7e-e1e61b94ed88 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.811622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 141/283] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/role_assignments?user.id=ab5c16b8f13c42e2b687998cfb3a66f7&scope.system=all&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:33.929136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1552e12-f4c3-40dc-b3db-f912d9ca0955 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.980798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e09810-0753-4c0b-b1a9-fd639c01f845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.981920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09810-0753-4c0b-b1a9-fd639c01f845 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.982443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09810-0753-4c0b-b1a9-fd639c01f845 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.982915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09810-0753-4c0b-b1a9-fd639c01f845 None admin] PATH_INFO: `/v3/users/93febba9030a47de93dc55826ef62a3d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.984255 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e09810-0753-4c0b-b1a9-fd639c01f845 None admin] RBAC: Authorizing `identity:get_user(user_id=93febba9030a47de93dc55826ef62a3d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.985945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e09810-0753-4c0b-b1a9-fd639c01f845 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.987751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 143/284] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/users/93febba9030a47de93dc55826ef62a3d => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:33.994010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94369e-b50d-42fc-8e7e-e1e61b94ed88 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:34.006251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bf59a-0a11-4e1e-bceb-d7e9be9c06b4 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:34.007468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bf59a-0a11-4e1e-bceb-d7e9be9c06b4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:34.007468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bf59a-0a11-4e1e-bceb-d7e9be9c06b4 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:34.008259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 144/285] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:12:34.013226 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c32757-c8f4-4ee5-91f7-7a7b5491ef62 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:34.013545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c32757-c8f4-4ee5-91f7-7a7b5491ef62 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:34.013822 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c32757-c8f4-4ee5-91f7-7a7b5491ef62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:34.020049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:34.021793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:34.035903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-20c32757-c8f4-4ee5-91f7-7a7b5491ef62 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:34.037088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-20c32757-c8f4-4ee5-91f7-7a7b5491ef62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:34.163412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 145/286] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:34.192520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94369e-b50d-42fc-8e7e-e1e61b94ed88 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:34.250162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453594b-87db-4cd5-9048-ee3de3cf2b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:34.250800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453594b-87db-4cd5-9048-ee3de3cf2b77 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:34.251080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453594b-87db-4cd5-9048-ee3de3cf2b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:34.251354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453594b-87db-4cd5-9048-ee3de3cf2b77 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:34.252509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453594b-87db-4cd5-9048-ee3de3cf2b77 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:34.254061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453594b-87db-4cd5-9048-ee3de3cf2b77 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:34.255709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 142/287] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:34.261539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c32757-c8f4-4ee5-91f7-7a7b5491ef62 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:34.406034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c32757-c8f4-4ee5-91f7-7a7b5491ef62 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:34.459202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b8ba5-06f1-4d6c-9770-134e50c4f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:34.459868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b8ba5-06f1-4d6c-9770-134e50c4f7a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:34.460142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b8ba5-06f1-4d6c-9770-134e50c4f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:34.460411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b8ba5-06f1-4d6c-9770-134e50c4f7a1 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808/users/93febba9030a47de93dc55826ef62a3d/roles/d529d6d058424562a72162b0c257851f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:34.462355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b8ba5-06f1-4d6c-9770-134e50c4f7a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61e52cca9584059bf484cabd52d2808, user_id=93febba9030a47de93dc55826ef62a3d, role_id=d529d6d058424562a72162b0c257851f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:34.463773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b8ba5-06f1-4d6c-9770-134e50c4f7a1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:34.465971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:34.570671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 146/288] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:34 2022] PUT /identity/v3/projects/d61e52cca9584059bf484cabd52d2808/users/93febba9030a47de93dc55826ef62a3d/roles/d529d6d058424562a72162b0c257851f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:34.596676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453594b-87db-4cd5-9048-ee3de3cf2b77 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:34.700195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:34.727610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453594b-87db-4cd5-9048-ee3de3cf2b77 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:34.770582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca74dc-a9b5-4993-84d7-2cdfc5867508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:34.771237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca74dc-a9b5-4993-84d7-2cdfc5867508 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:34.771511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca74dc-a9b5-4993-84d7-2cdfc5867508 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:34.771782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca74dc-a9b5-4993-84d7-2cdfc5867508 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:34.783369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca74dc-a9b5-4993-84d7-2cdfc5867508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-35ca74dc-a9b5-4993-84d7-2cdfc5867508 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:34.789332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:34.790318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:34.791190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 143/289] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:34.797897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b8ba5-06f1-4d6c-9770-134e50c4f7a1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:34.914034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b8ba5-06f1-4d6c-9770-134e50c4f7a1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:34.953401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64abd92-16f8-4488-98ec-d4365a7734f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:34.953964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64abd92-16f8-4488-98ec-d4365a7734f8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:34.954298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64abd92-16f8-4488-98ec-d4365a7734f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:34.954569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64abd92-16f8-4488-98ec-d4365a7734f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:34.955227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64abd92-16f8-4488-98ec-d4365a7734f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:34.955523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64abd92-16f8-4488-98ec-d4365a7734f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:34.957147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64abd92-16f8-4488-98ec-d4365a7734f8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:34.966010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 147/290] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:34.971330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca74dc-a9b5-4993-84d7-2cdfc5867508 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:35.098381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca74dc-a9b5-4993-84d7-2cdfc5867508 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:35.132904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6bc07-36c0-441b-abd0-f6bcd4a0cf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:35.133609 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6bc07-36c0-441b-abd0-f6bcd4a0cf51 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:35.133889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6bc07-36c0-441b-abd0-f6bcd4a0cf51 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:35.134155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6bc07-36c0-441b-abd0-f6bcd4a0cf51 None admin] PATH_INFO: `/v3/users/f461d4cb769f4b658b434a594e78c32c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:35.135282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6bc07-36c0-441b-abd0-f6bcd4a0cf51 None admin] RBAC: Authorizing `identity:get_user(user_id=f461d4cb769f4b658b434a594e78c32c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:35.136797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6bc07-36c0-441b-abd0-f6bcd4a0cf51 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:35.138399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 144/291] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:34 2022] GET /identity/v3/users/f461d4cb769f4b658b434a594e78c32c => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:35.144691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64abd92-16f8-4488-98ec-d4365a7734f8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:35.209935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:35.253838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64abd92-16f8-4488-98ec-d4365a7734f8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:35.308618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d5342-af60-48d2-8f8c-f8195b79e63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:35.309303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d5342-af60-48d2-8f8c-f8195b79e63f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:35.309611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d5342-af60-48d2-8f8c-f8195b79e63f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:35.309884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d5342-af60-48d2-8f8c-f8195b79e63f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:35.404025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d5342-af60-48d2-8f8c-f8195b79e63f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:35.405504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d5342-af60-48d2-8f8c-f8195b79e63f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:35.485937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:35.545189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 148/292] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:35.550495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6bc07-36c0-441b-abd0-f6bcd4a0cf51 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:35.558048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:35.712438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6bc07-36c0-441b-abd0-f6bcd4a0cf51 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:35.765731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60a921-3dfc-4f89-a421-77502d4fd7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:35.766278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60a921-3dfc-4f89-a421-77502d4fd7ec None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:35.766551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60a921-3dfc-4f89-a421-77502d4fd7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:35.766818 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60a921-3dfc-4f89-a421-77502d4fd7ec None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:35.768129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60a921-3dfc-4f89-a421-77502d4fd7ec None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:35.770710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60a921-3dfc-4f89-a421-77502d4fd7ec None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:35.772635 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 145/293] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:35 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:35.778458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d5342-af60-48d2-8f8c-f8195b79e63f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:35.936390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d5342-af60-48d2-8f8c-f8195b79e63f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:35.983089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f4a39-697c-4dae-8d33-6b188d120fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:35.983713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f4a39-697c-4dae-8d33-6b188d120fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:35.983993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f4a39-697c-4dae-8d33-6b188d120fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:35.984266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f4a39-697c-4dae-8d33-6b188d120fd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:35.985002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f4a39-697c-4dae-8d33-6b188d120fd6 None admin] RBAC: Adding query filter params (role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, scope.project.id=d61e52cca9584059bf484cabd52d2808, user.id=f461d4cb769f4b658b434a594e78c32c) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:35.985309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f4a39-697c-4dae-8d33-6b188d120fd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:35.986767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f4a39-697c-4dae-8d33-6b188d120fd6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:35.998392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 149/294] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:35 2022] GET /identity/v3/role_assignments?user.id=f461d4cb769f4b658b434a594e78c32c&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:36.028317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea674-89bb-44b9-8984-b998cc0dc66f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.028938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea674-89bb-44b9-8984-b998cc0dc66f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.029406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea674-89bb-44b9-8984-b998cc0dc66f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.031197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 146/295] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:36.037254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f44d57-dbb2-47c1-ac68-c8603d398fb1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.037618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f44d57-dbb2-47c1-ac68-c8603d398fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.037849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f44d57-dbb2-47c1-ac68-c8603d398fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.068536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-63f44d57-dbb2-47c1-ac68-c8603d398fb1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:36.070164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-63f44d57-dbb2-47c1-ac68-c8603d398fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:36.092497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:36.127070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb19bfe-a4bd-40b0-b33d-96d8e00bbeee None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.127407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb19bfe-a4bd-40b0-b33d-96d8e00bbeee None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.127619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb19bfe-a4bd-40b0-b33d-96d8e00bbeee None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.129141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 147/296] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:36.138001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d243fcd-5437-4021-affb-9cd3b75daf06 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.138309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d243fcd-5437-4021-affb-9cd3b75daf06 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.138551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d243fcd-5437-4021-affb-9cd3b75daf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.160135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2d243fcd-5437-4021-affb-9cd3b75daf06 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:36.161422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2d243fcd-5437-4021-affb-9cd3b75daf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:36.186978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 150/297] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:36.259933 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 148/298] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:36.355679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13899d49-3cff-40ff-8e60-c10bfff212b2 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.356541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13899d49-3cff-40ff-8e60-c10bfff212b2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.356958 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13899d49-3cff-40ff-8e60-c10bfff212b2 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.358476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 151/299] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:36.363687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11d5f5-67af-427d-8ad1-6dbe0610f3e6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.363969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11d5f5-67af-427d-8ad1-6dbe0610f3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.364247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11d5f5-67af-427d-8ad1-6dbe0610f3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.370653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:36.384513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cf11d5f5-67af-427d-8ad1-6dbe0610f3e6 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:36.393642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cf11d5f5-67af-427d-8ad1-6dbe0610f3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:36.464736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 149/300] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:36.509503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13899d49-3cff-40ff-8e60-c10bfff212b2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:36.556763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11d5f5-67af-427d-8ad1-6dbe0610f3e6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:36.621138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13899d49-3cff-40ff-8e60-c10bfff212b2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:36.666798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a832f-7e9f-450e-b53d-32a19bba80d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:36.667436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a832f-7e9f-450e-b53d-32a19bba80d1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.667712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a832f-7e9f-450e-b53d-32a19bba80d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.667974 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a832f-7e9f-450e-b53d-32a19bba80d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.682802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11d5f5-67af-427d-8ad1-6dbe0610f3e6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:36.682954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a832f-7e9f-450e-b53d-32a19bba80d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-fb5a832f-7e9f-450e-b53d-32a19bba80d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:36.688673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:36.689603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:36.690417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:36.698868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 152/301] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:36.700318 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a832f-7e9f-450e-b53d-32a19bba80d1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:36.716201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:36.729429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:36.736419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09802794-b4e4-4a38-b94c-47b0b15f6944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:36.737089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09802794-b4e4-4a38-b94c-47b0b15f6944 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.737375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09802794-b4e4-4a38-b94c-47b0b15f6944 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.737672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09802794-b4e4-4a38-b94c-47b0b15f6944 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.739381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09802794-b4e4-4a38-b94c-47b0b15f6944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:36.741004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09802794-b4e4-4a38-b94c-47b0b15f6944 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:36.743179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 150/302] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:36.751298 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09802794-b4e4-4a38-b94c-47b0b15f6944 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:36.815109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a832f-7e9f-450e-b53d-32a19bba80d1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:36.849907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e662a-9a65-4004-8638-ce907d1565fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:36.850519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e662a-9a65-4004-8638-ce907d1565fc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.850795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e662a-9a65-4004-8638-ce907d1565fc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.851064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e662a-9a65-4004-8638-ce907d1565fc None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.851723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e662a-9a65-4004-8638-ce907d1565fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:36.852022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e662a-9a65-4004-8638-ce907d1565fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:36.853389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e662a-9a65-4004-8638-ce907d1565fc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:36.862111 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 153/303] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:36.863510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e662a-9a65-4004-8638-ce907d1565fc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:36.869922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09802794-b4e4-4a38-b94c-47b0b15f6944 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:36.906923 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7345b4-79be-43f7-8bdd-6dc1ee7ded3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:36.907600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7345b4-79be-43f7-8bdd-6dc1ee7ded3f None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.907954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7345b4-79be-43f7-8bdd-6dc1ee7ded3f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.908280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7345b4-79be-43f7-8bdd-6dc1ee7ded3f None admin] PATH_INFO: `/v3/groups` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.908908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7345b4-79be-43f7-8bdd-6dc1ee7ded3f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:36.910537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7345b4-79be-43f7-8bdd-6dc1ee7ded3f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:36.936680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:36.967161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e662a-9a65-4004-8638-ce907d1565fc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:36.969401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 151/304] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Sep 1 05:12:36 2022] POST /identity/v3/groups => generated 229 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:36.970424 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7345b4-79be-43f7-8bdd-6dc1ee7ded3f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:37.027581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae90482-a587-4ea4-b804-23d66ac85e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:37.028330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae90482-a587-4ea4-b804-23d66ac85e80 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:37.028615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae90482-a587-4ea4-b804-23d66ac85e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:37.028888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae90482-a587-4ea4-b804-23d66ac85e80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:37.041987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae90482-a587-4ea4-b804-23d66ac85e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0ae90482-a587-4ea4-b804-23d66ac85e80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:37.049359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:37.050774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:37.051568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 154/305] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:37.052244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8c7c1-1291-41c5-bf1a-8709eed7a829 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:37.052844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8c7c1-1291-41c5-bf1a-8709eed7a829 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:37.052844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8c7c1-1291-41c5-bf1a-8709eed7a829 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:37.054157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:37.054283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 155/306] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:37.059102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8c7c1-1291-41c5-bf1a-8709eed7a829 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:37.110130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7345b4-79be-43f7-8bdd-6dc1ee7ded3f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:37.154645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca2646-c71f-48fd-a3d1-c1c770dda69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:37.155279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca2646-c71f-48fd-a3d1-c1c770dda69e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:37.155555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca2646-c71f-48fd-a3d1-c1c770dda69e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:37.155828 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca2646-c71f-48fd-a3d1-c1c770dda69e None admin] PATH_INFO: `/v3/users/25cfddf078b84199982756003b82dc05` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:37.172983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca2646-c71f-48fd-a3d1-c1c770dda69e None admin] RBAC: Authorizing `identity:get_user(user_id=25cfddf078b84199982756003b82dc05)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:37.174498 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca2646-c71f-48fd-a3d1-c1c770dda69e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:37.176566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 152/307] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:36 2022] GET /identity/v3/users/25cfddf078b84199982756003b82dc05 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:37.177654 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7db100-8012-4839-96ef-1570560bb64a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:37.177988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7db100-8012-4839-96ef-1570560bb64a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:37.178267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7db100-8012-4839-96ef-1570560bb64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:37.182058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8c7c1-1291-41c5-bf1a-8709eed7a829 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:37.197616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3a7db100-8012-4839-96ef-1570560bb64a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:37.198958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3a7db100-8012-4839-96ef-1570560bb64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:37.217094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:37.221945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e049a-530f-4eea-9f02-2670de7f5611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:37.222776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e049a-530f-4eea-9f02-2670de7f5611 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:37.223258 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e049a-530f-4eea-9f02-2670de7f5611 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:37.223535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e049a-530f-4eea-9f02-2670de7f5611 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:37.224197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e049a-530f-4eea-9f02-2670de7f5611 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:37.224539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e049a-530f-4eea-9f02-2670de7f5611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:37.225980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e049a-530f-4eea-9f02-2670de7f5611 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:37.234381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 156/308] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:37.235182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e049a-530f-4eea-9f02-2670de7f5611 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:37.279188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 153/309] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:37.280379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7db100-8012-4839-96ef-1570560bb64a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:37.352247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:37.364524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e049a-530f-4eea-9f02-2670de7f5611 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:37.403102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b18e86-2fd7-4819-a514-730612b4e4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:37.404019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b18e86-2fd7-4819-a514-730612b4e4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:37.404568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b18e86-2fd7-4819-a514-730612b4e4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:37.404949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b18e86-2fd7-4819-a514-730612b4e4b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:37.405820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b18e86-2fd7-4819-a514-730612b4e4b2 None admin] RBAC: Adding query filter params (role.id=d529d6d058424562a72162b0c257851f, user.id=25cfddf078b84199982756003b82dc05, scope.system=all) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:37.406223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b18e86-2fd7-4819-a514-730612b4e4b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:37.407878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b18e86-2fd7-4819-a514-730612b4e4b2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:37.414266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 157/310] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 1 05:12:37 2022] GET /identity/v3/role_assignments?user.id=25cfddf078b84199982756003b82dc05&scope.system=all&role.id=d529d6d058424562a72162b0c257851f => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:37.418973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7db100-8012-4839-96ef-1570560bb64a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:37.459381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8793b-3983-402a-87aa-c6eff6d59b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:37.460326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8793b-3983-402a-87aa-c6eff6d59b4f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:37.460771 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8793b-3983-402a-87aa-c6eff6d59b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:37.461249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8793b-3983-402a-87aa-c6eff6d59b4f None admin] PATH_INFO: `/v3/users/ab5c16b8f13c42e2b687998cfb3a66f7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:37.462795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8793b-3983-402a-87aa-c6eff6d59b4f None admin] RBAC: Authorizing `identity:get_user(user_id=ab5c16b8f13c42e2b687998cfb3a66f7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:37.464453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8793b-3983-402a-87aa-c6eff6d59b4f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:37.466353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 154/311] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:37 2022] GET /identity/v3/users/ab5c16b8f13c42e2b687998cfb3a66f7 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:37.473164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b18e86-2fd7-4819-a514-730612b4e4b2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:37.549040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:37.598339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b18e86-2fd7-4819-a514-730612b4e4b2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:37.611370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8793b-3983-402a-87aa-c6eff6d59b4f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:37.648235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6ba87-8130-402c-972a-178efea1e0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:37.648902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6ba87-8130-402c-972a-178efea1e0fd None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:37.649184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6ba87-8130-402c-972a-178efea1e0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:37.649477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6ba87-8130-402c-972a-178efea1e0fd None admin] PATH_INFO: `/v3/system/users/ab5c16b8f13c42e2b687998cfb3a66f7/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:37.651145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6ba87-8130-402c-972a-178efea1e0fd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ab5c16b8f13c42e2b687998cfb3a66f7, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:37.652527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6ba87-8130-402c-972a-178efea1e0fd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:37.731157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8793b-3983-402a-87aa-c6eff6d59b4f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:37.753488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 158/312] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Sep 1 05:12:37 2022] PUT /identity/v3/system/users/ab5c16b8f13c42e2b687998cfb3a66f7/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:37.757852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:37.778845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41b6a8-bf7c-47c8-b39d-ccc0720be00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:37.779494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41b6a8-bf7c-47c8-b39d-ccc0720be00a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:37.779825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41b6a8-bf7c-47c8-b39d-ccc0720be00a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:37.780103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41b6a8-bf7c-47c8-b39d-ccc0720be00a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:37.793474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41b6a8-bf7c-47c8-b39d-ccc0720be00a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8e41b6a8-bf7c-47c8-b39d-ccc0720be00a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:37.798820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:37.799744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:37.802194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 155/313] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:37.806775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6ba87-8130-402c-972a-178efea1e0fd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:37.927612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6ba87-8130-402c-972a-178efea1e0fd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:37.952773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:37.965090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239b1e7-7111-4513-a735-560bd6b6d367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:37.970009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b1e7-7111-4513-a735-560bd6b6d367 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:37.970009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b1e7-7111-4513-a735-560bd6b6d367 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:37.970009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b1e7-7111-4513-a735-560bd6b6d367 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:37.970009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239b1e7-7111-4513-a735-560bd6b6d367 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:37.970009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239b1e7-7111-4513-a735-560bd6b6d367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:37.970009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239b1e7-7111-4513-a735-560bd6b6d367 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:37.979433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 159/314] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:37.985116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41b6a8-bf7c-47c8-b39d-ccc0720be00a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:38.099771 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41b6a8-bf7c-47c8-b39d-ccc0720be00a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:38.137269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afcb9e-644f-44f6-9f4a-b3cb94951529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:38.138259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afcb9e-644f-44f6-9f4a-b3cb94951529 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:38.138721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afcb9e-644f-44f6-9f4a-b3cb94951529 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:38.139169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afcb9e-644f-44f6-9f4a-b3cb94951529 None admin] PATH_INFO: `/v3/users/93febba9030a47de93dc55826ef62a3d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:38.140583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afcb9e-644f-44f6-9f4a-b3cb94951529 None admin] RBAC: Authorizing `identity:get_user(user_id=93febba9030a47de93dc55826ef62a3d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:38.142303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afcb9e-644f-44f6-9f4a-b3cb94951529 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:38.145196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 156/315] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:37 2022] GET /identity/v3/users/93febba9030a47de93dc55826ef62a3d => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:38.150768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239b1e7-7111-4513-a735-560bd6b6d367 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:38.257242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239b1e7-7111-4513-a735-560bd6b6d367 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:38.294190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee4a2f-0aef-4e6e-a91d-7b8e6ad28464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:38.295404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee4a2f-0aef-4e6e-a91d-7b8e6ad28464 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:38.295766 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee4a2f-0aef-4e6e-a91d-7b8e6ad28464 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:38.296042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee4a2f-0aef-4e6e-a91d-7b8e6ad28464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:38.352272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:38.369271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee4a2f-0aef-4e6e-a91d-7b8e6ad28464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:38.370826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee4a2f-0aef-4e6e-a91d-7b8e6ad28464 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:38.475333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 160/316] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:38.482579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afcb9e-644f-44f6-9f4a-b3cb94951529 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:38.537351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:38.596598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afcb9e-644f-44f6-9f4a-b3cb94951529 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:38.638138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1a653-251e-4688-b88e-011479f7203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:38.638920 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a653-251e-4688-b88e-011479f7203a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:38.639095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a653-251e-4688-b88e-011479f7203a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:38.639364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a653-251e-4688-b88e-011479f7203a None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:38.640503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1a653-251e-4688-b88e-011479f7203a None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:38.642008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1a653-251e-4688-b88e-011479f7203a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:38.643992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 157/317] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:38 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:38.649677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee4a2f-0aef-4e6e-a91d-7b8e6ad28464 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:38.738198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:38.754606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee4a2f-0aef-4e6e-a91d-7b8e6ad28464 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:38.795242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6a5e9-ecf0-42fc-a61f-5bbc3b2e8b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:38.795770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a5e9-ecf0-42fc-a61f-5bbc3b2e8b27 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:38.796044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a5e9-ecf0-42fc-a61f-5bbc3b2e8b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:38.796312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a5e9-ecf0-42fc-a61f-5bbc3b2e8b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:38.797040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6a5e9-ecf0-42fc-a61f-5bbc3b2e8b27 None admin] RBAC: Adding query filter params (role.id=d529d6d058424562a72162b0c257851f, scope.project.id=d61e52cca9584059bf484cabd52d2808, user.id=93febba9030a47de93dc55826ef62a3d) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:38.797344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6a5e9-ecf0-42fc-a61f-5bbc3b2e8b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:38.798801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6a5e9-ecf0-42fc-a61f-5bbc3b2e8b27 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:38.805691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 161/318] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:38 2022] GET /identity/v3/role_assignments?user.id=93febba9030a47de93dc55826ef62a3d&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=d529d6d058424562a72162b0c257851f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:39.025435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b03980-cdb4-41bf-b94d-aa61b87f2dcd None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.025812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b03980-cdb4-41bf-b94d-aa61b87f2dcd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.026016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b03980-cdb4-41bf-b94d-aa61b87f2dcd None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.027550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 158/319] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:39.032508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26395a-a8eb-4163-843a-bd07bcd4c446 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.032796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26395a-a8eb-4163-843a-bd07bcd4c446 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.033070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26395a-a8eb-4163-843a-bd07bcd4c446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.052735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6e26395a-a8eb-4163-843a-bd07bcd4c446 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:39.053883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6e26395a-a8eb-4163-843a-bd07bcd4c446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:39.122805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 162/320] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:39.404276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b03980-cdb4-41bf-b94d-aa61b87f2dcd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:39.436310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc6233-6ed0-40fc-ac00-624bb59d8576 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.437157 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc6233-6ed0-40fc-ac00-624bb59d8576 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.437431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc6233-6ed0-40fc-ac00-624bb59d8576 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.439279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 163/321] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:12:39.444449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138c8f-fc18-403c-b78e-865afc21e212 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.444732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138c8f-fc18-403c-b78e-865afc21e212 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.445001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138c8f-fc18-403c-b78e-865afc21e212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.464468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-05138c8f-fc18-403c-b78e-865afc21e212 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:39.465686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-05138c8f-fc18-403c-b78e-865afc21e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:39.521421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b03980-cdb4-41bf-b94d-aa61b87f2dcd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:39.540315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:39.542610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 164/322] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:39.562342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3835414-f51e-4855-abfd-78bf2400e2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:39.563016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3835414-f51e-4855-abfd-78bf2400e2aa None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.563257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3835414-f51e-4855-abfd-78bf2400e2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.563903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3835414-f51e-4855-abfd-78bf2400e2aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.565638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3835414-f51e-4855-abfd-78bf2400e2aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:39.567186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3835414-f51e-4855-abfd-78bf2400e2aa None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:39.569494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 159/323] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:39.574294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138c8f-fc18-403c-b78e-865afc21e212 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:39.589832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:39.686564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138c8f-fc18-403c-b78e-865afc21e212 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:39.730378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa918930-d57f-4065-b5ff-98522bb22fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:39.731002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa918930-d57f-4065-b5ff-98522bb22fca None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.731284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa918930-d57f-4065-b5ff-98522bb22fca None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.731560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa918930-d57f-4065-b5ff-98522bb22fca None admin] PATH_INFO: `/v3/groups` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.732190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa918930-d57f-4065-b5ff-98522bb22fca None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:39.733665 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa918930-d57f-4065-b5ff-98522bb22fca None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:39.805490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 165/324] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Sep 1 05:12:39 2022] POST /identity/v3/groups => generated 226 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:39.827867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3835414-f51e-4855-abfd-78bf2400e2aa None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:39.831820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af47c00-6c1c-48a2-83c7-5ac1928672e1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.832126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af47c00-6c1c-48a2-83c7-5ac1928672e1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.832398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af47c00-6c1c-48a2-83c7-5ac1928672e1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.833890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 166/325] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:39.839441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0af52-ab4c-4a78-bbea-963df55e45b4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.839731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0af52-ab4c-4a78-bbea-963df55e45b4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.840008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0af52-ab4c-4a78-bbea-963df55e45b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.859556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6ff0af52-ab4c-4a78-bbea-963df55e45b4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:39.860792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6ff0af52-ab4c-4a78-bbea-963df55e45b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:39.931873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:39.944275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 167/326] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:39.947904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3835414-f51e-4855-abfd-78bf2400e2aa None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:39.990558 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cbca-8319-4cd4-bf48-cf3d8dfc7600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:39.991199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cbca-8319-4cd4-bf48-cf3d8dfc7600 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.991480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cbca-8319-4cd4-bf48-cf3d8dfc7600 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.991773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cbca-8319-4cd4-bf48-cf3d8dfc7600 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:40.002315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46cbca-8319-4cd4-bf48-cf3d8dfc7600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-aa46cbca-8319-4cd4-bf48-cf3d8dfc7600 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:40.007671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:40.008758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:40.009732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 160/327] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:40.014807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0af52-ab4c-4a78-bbea-963df55e45b4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:40.124605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0af52-ab4c-4a78-bbea-963df55e45b4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:40.161251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94242ad-ade5-4678-baa2-484da2a78c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:40.161891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94242ad-ade5-4678-baa2-484da2a78c56 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:40.162166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94242ad-ade5-4678-baa2-484da2a78c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:40.162435 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94242ad-ade5-4678-baa2-484da2a78c56 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:40.163098 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94242ad-ade5-4678-baa2-484da2a78c56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:40.163396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94242ad-ade5-4678-baa2-484da2a78c56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:40.164753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94242ad-ade5-4678-baa2-484da2a78c56 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:40.177555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 168/328] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:40.180025 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cbca-8319-4cd4-bf48-cf3d8dfc7600 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:40.226190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:40.233604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94242ad-ade5-4678-baa2-484da2a78c56 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:40.309289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:40.312123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cbca-8319-4cd4-bf48-cf3d8dfc7600 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:40.357642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd66c-50a2-4cc1-ad36-8ab7650d73ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:40.358711 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd66c-50a2-4cc1-ad36-8ab7650d73ac None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:40.359288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd66c-50a2-4cc1-ad36-8ab7650d73ac None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:40.359568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd66c-50a2-4cc1-ad36-8ab7650d73ac None admin] PATH_INFO: `/v3/users/25cfddf078b84199982756003b82dc05` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:40.360916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cd66c-50a2-4cc1-ad36-8ab7650d73ac None admin] RBAC: Authorizing `identity:get_user(user_id=25cfddf078b84199982756003b82dc05)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:40.363631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cd66c-50a2-4cc1-ad36-8ab7650d73ac None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:40.364436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94242ad-ade5-4678-baa2-484da2a78c56 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:40.365436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 161/329] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:40 2022] GET /identity/v3/users/25cfddf078b84199982756003b82dc05 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:40.373187 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd66c-50a2-4cc1-ad36-8ab7650d73ac None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:40.416102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba070a2-713b-4479-8d18-709a008485a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:40.416930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba070a2-713b-4479-8d18-709a008485a8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:40.417221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba070a2-713b-4479-8d18-709a008485a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:40.417526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba070a2-713b-4479-8d18-709a008485a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:40.432861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba070a2-713b-4479-8d18-709a008485a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3ba070a2-713b-4479-8d18-709a008485a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:40.440051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:40.442210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:40.443130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:40.443734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 169/330] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:40.449073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba070a2-713b-4479-8d18-709a008485a8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:40.503938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd66c-50a2-4cc1-ad36-8ab7650d73ac None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:40.516168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:40.559211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643360ca-3102-4b34-bb10-a4c88e05bcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:40.559877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643360ca-3102-4b34-bb10-a4c88e05bcd7 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:40.560155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643360ca-3102-4b34-bb10-a4c88e05bcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:40.560426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643360ca-3102-4b34-bb10-a4c88e05bcd7 None admin] PATH_INFO: `/v3/system/users/25cfddf078b84199982756003b82dc05/roles/d529d6d058424562a72162b0c257851f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:40.564552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643360ca-3102-4b34-bb10-a4c88e05bcd7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=25cfddf078b84199982756003b82dc05, role_id=d529d6d058424562a72162b0c257851f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:40.566251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643360ca-3102-4b34-bb10-a4c88e05bcd7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:40.583590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba070a2-713b-4479-8d18-709a008485a8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:40.608434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:40.623714 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11599d8-2f75-451d-9d03-debc8ef2e70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:40.624339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11599d8-2f75-451d-9d03-debc8ef2e70c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:40.624620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11599d8-2f75-451d-9d03-debc8ef2e70c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:40.624894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11599d8-2f75-451d-9d03-debc8ef2e70c None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:40.625645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11599d8-2f75-451d-9d03-debc8ef2e70c None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:40.626055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11599d8-2f75-451d-9d03-debc8ef2e70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:40.627424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11599d8-2f75-451d-9d03-debc8ef2e70c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:40.638072 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 170/331] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:40.643634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11599d8-2f75-451d-9d03-debc8ef2e70c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:40.645005 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 162/332] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Sep 1 05:12:40 2022] PUT /identity/v3/system/users/25cfddf078b84199982756003b82dc05/roles/d529d6d058424562a72162b0c257851f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:40.768310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11599d8-2f75-451d-9d03-debc8ef2e70c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:40.807721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b93060-ce3f-415a-a8fe-8ab1b34dc8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:40.808373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93060-ce3f-415a-a8fe-8ab1b34dc8ff None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:40.808648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93060-ce3f-415a-a8fe-8ab1b34dc8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:40.808924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93060-ce3f-415a-a8fe-8ab1b34dc8ff None admin] PATH_INFO: `/v3/users/ab5c16b8f13c42e2b687998cfb3a66f7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:40.810159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b93060-ce3f-415a-a8fe-8ab1b34dc8ff None admin] RBAC: Authorizing `identity:get_user(user_id=ab5c16b8f13c42e2b687998cfb3a66f7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:40.811628 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b93060-ce3f-415a-a8fe-8ab1b34dc8ff None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:40.813404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 171/333] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:40 2022] GET /identity/v3/users/ab5c16b8f13c42e2b687998cfb3a66f7 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:40.820768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643360ca-3102-4b34-bb10-a4c88e05bcd7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:40.930427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643360ca-3102-4b34-bb10-a4c88e05bcd7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:40.968663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9e023-06a8-4c5f-acc3-86e8b30facb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:40.969269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9e023-06a8-4c5f-acc3-86e8b30facb4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:40.969590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9e023-06a8-4c5f-acc3-86e8b30facb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:40.969860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9e023-06a8-4c5f-acc3-86e8b30facb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:40.970583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9e023-06a8-4c5f-acc3-86e8b30facb4 None admin] RBAC: Adding query filter params (role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, user.id=ab5c16b8f13c42e2b687998cfb3a66f7, scope.system=all) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:40.970950 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9e023-06a8-4c5f-acc3-86e8b30facb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:40.973265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9e023-06a8-4c5f-acc3-86e8b30facb4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:40.997748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 163/334] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 1 05:12:40 2022] GET /identity/v3/role_assignments?user.id=ab5c16b8f13c42e2b687998cfb3a66f7&scope.system=all&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:42.211216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3d892-3e2e-4c6f-aeef-2899c8d56f84 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:42.211216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3d892-3e2e-4c6f-aeef-2899c8d56f84 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:42.211216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3d892-3e2e-4c6f-aeef-2899c8d56f84 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:42.212608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 172/335] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:42.216877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26300240-b3a5-4dc4-b603-fd8c593a56d6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:42.220435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26300240-b3a5-4dc4-b603-fd8c593a56d6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:42.220435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26300240-b3a5-4dc4-b603-fd8c593a56d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:42.244928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-26300240-b3a5-4dc4-b603-fd8c593a56d6 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:42.246638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-26300240-b3a5-4dc4-b603-fd8c593a56d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:42.279404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:42.333820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 164/336] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:42.593176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6be8ee-ffca-4ced-aeac-258726e370dc None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:42.593521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6be8ee-ffca-4ced-aeac-258726e370dc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:42.593770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6be8ee-ffca-4ced-aeac-258726e370dc None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:42.595497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 173/337] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:42.603305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44f676-1ddd-4222-8288-ce8ffe51da99 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:42.603593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44f676-1ddd-4222-8288-ce8ffe51da99 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:42.603862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44f676-1ddd-4222-8288-ce8ffe51da99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:42.629289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5a44f676-1ddd-4222-8288-ce8ffe51da99 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:42.630509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5a44f676-1ddd-4222-8288-ce8ffe51da99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:42.733896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 165/338] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:43.025953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6be8ee-ffca-4ced-aeac-258726e370dc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:43.091347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44f676-1ddd-4222-8288-ce8ffe51da99 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:43.095328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:43.186396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6be8ee-ffca-4ced-aeac-258726e370dc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:43.247792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:43.258560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec9d78-b9ac-4daf-a952-3b94ce91fd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:43.259207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec9d78-b9ac-4daf-a952-3b94ce91fd93 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:43.259485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec9d78-b9ac-4daf-a952-3b94ce91fd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:43.259753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec9d78-b9ac-4daf-a952-3b94ce91fd93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:43.262283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44f676-1ddd-4222-8288-ce8ffe51da99 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:43.286237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec9d78-b9ac-4daf-a952-3b94ce91fd93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-35ec9d78-b9ac-4daf-a952-3b94ce91fd93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:43.292310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:43.293298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:43.294195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:43.296526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 174/339] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:43.298136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874eb6d-20ed-4d7f-a8ce-0590cf415aab None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:43.298425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874eb6d-20ed-4d7f-a8ce-0590cf415aab None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:43.298698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874eb6d-20ed-4d7f-a8ce-0590cf415aab None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:43.300079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 175/340] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:43.303154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874eb6d-20ed-4d7f-a8ce-0590cf415aab None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:43.319631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18ad76-e0bc-45de-80e2-c1d37bf440ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:43.320330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ad76-e0bc-45de-80e2-c1d37bf440ef None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:43.320614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ad76-e0bc-45de-80e2-c1d37bf440ef None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:43.320885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ad76-e0bc-45de-80e2-c1d37bf440ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:43.344619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18ad76-e0bc-45de-80e2-c1d37bf440ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-dd18ad76-e0bc-45de-80e2-c1d37bf440ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:43.350353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:43.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:43.352373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:43.354648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 166/341] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:43.360295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3081ce-ca13-4da8-b740-4e636554cb45 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:43.360295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3081ce-ca13-4da8-b740-4e636554cb45 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:43.360295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3081ce-ca13-4da8-b740-4e636554cb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:43.382974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cc3081ce-ca13-4da8-b740-4e636554cb45 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:43.384177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cc3081ce-ca13-4da8-b740-4e636554cb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:43.432391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874eb6d-20ed-4d7f-a8ce-0590cf415aab None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:43.444036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:43.459853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 167/342] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:43.461225 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d7b29-7c49-4917-aae8-d41c90e1a440 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:43.461561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d7b29-7c49-4917-aae8-d41c90e1a440 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:43.461823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d7b29-7c49-4917-aae8-d41c90e1a440 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:43.463425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 168/343] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:43.464160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d7b29-7c49-4917-aae8-d41c90e1a440 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:43.478305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15551472-7e5e-4214-82d3-0cfbfc4f0f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:43.478934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15551472-7e5e-4214-82d3-0cfbfc4f0f6a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:43.479482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15551472-7e5e-4214-82d3-0cfbfc4f0f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:43.479755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15551472-7e5e-4214-82d3-0cfbfc4f0f6a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:43.480422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15551472-7e5e-4214-82d3-0cfbfc4f0f6a None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:43.480723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15551472-7e5e-4214-82d3-0cfbfc4f0f6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:43.482121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15551472-7e5e-4214-82d3-0cfbfc4f0f6a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:43.491988 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 176/344] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:43.493281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d23114-cdac-4a7f-bfcb-ced82cfc7654 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:43.493605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d23114-cdac-4a7f-bfcb-ced82cfc7654 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:43.493878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d23114-cdac-4a7f-bfcb-ced82cfc7654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:43.520850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-57d23114-cdac-4a7f-bfcb-ced82cfc7654 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:43.522083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-57d23114-cdac-4a7f-bfcb-ced82cfc7654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:43.604875 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 177/345] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:43.606961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d23114-cdac-4a7f-bfcb-ced82cfc7654 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:43.607461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:43.630319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d7b29-7c49-4917-aae8-d41c90e1a440 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:43.728246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71645e6-2cd4-474b-84b8-f6ad04df06e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:43.728874 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71645e6-2cd4-474b-84b8-f6ad04df06e9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:43.729149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71645e6-2cd4-474b-84b8-f6ad04df06e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:43.729416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71645e6-2cd4-474b-84b8-f6ad04df06e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:43.730120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71645e6-2cd4-474b-84b8-f6ad04df06e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:43.730420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71645e6-2cd4-474b-84b8-f6ad04df06e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:43.731807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71645e6-2cd4-474b-84b8-f6ad04df06e9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:43.749273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 169/346] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:43.755888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71645e6-2cd4-474b-84b8-f6ad04df06e9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:43.798006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d23114-cdac-4a7f-bfcb-ced82cfc7654 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:43.799708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:43.844578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a27da3-cbf4-41f8-aa65-1c29d44bf342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:43.845675 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a27da3-cbf4-41f8-aa65-1c29d44bf342 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:43.845968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a27da3-cbf4-41f8-aa65-1c29d44bf342 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:43.846244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a27da3-cbf4-41f8-aa65-1c29d44bf342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:43.973229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71645e6-2cd4-474b-84b8-f6ad04df06e9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:43.976657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a27da3-cbf4-41f8-aa65-1c29d44bf342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:43.978156 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a27da3-cbf4-41f8-aa65-1c29d44bf342 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:44.020414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919abfd7-add5-4030-895a-0e6d97e6ffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:44.021029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919abfd7-add5-4030-895a-0e6d97e6ffab None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.021310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919abfd7-add5-4030-895a-0e6d97e6ffab None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.021619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919abfd7-add5-4030-895a-0e6d97e6ffab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.036161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:44.109546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 178/347] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:44.110411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a27da3-cbf4-41f8-aa65-1c29d44bf342 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:44.111282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919abfd7-add5-4030-895a-0e6d97e6ffab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:44.113521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:44.118276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919abfd7-add5-4030-895a-0e6d97e6ffab None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:44.221244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a27da3-cbf4-41f8-aa65-1c29d44bf342 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:44.228517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 170/348] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:12:44.229306 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919abfd7-add5-4030-895a-0e6d97e6ffab None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:44.259772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de874d60-d23f-4c6a-8a53-80173ad56657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:44.260518 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de874d60-d23f-4c6a-8a53-80173ad56657 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.260807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de874d60-d23f-4c6a-8a53-80173ad56657 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.261083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de874d60-d23f-4c6a-8a53-80173ad56657 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.272622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de874d60-d23f-4c6a-8a53-80173ad56657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:44.277083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-de874d60-d23f-4c6a-8a53-80173ad56657 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:44.278096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:44.278987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:44.280389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 179/349] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:44.281080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de874d60-d23f-4c6a-8a53-80173ad56657 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:44.340141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919abfd7-add5-4030-895a-0e6d97e6ffab None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:44.348812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:44.377084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b43763-afaf-4f36-a8ef-c03a3cd70eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:44.377765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b43763-afaf-4f36-a8ef-c03a3cd70eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.378745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b43763-afaf-4f36-a8ef-c03a3cd70eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.379195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b43763-afaf-4f36-a8ef-c03a3cd70eb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.391864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de874d60-d23f-4c6a-8a53-80173ad56657 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:44.392164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b43763-afaf-4f36-a8ef-c03a3cd70eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-a9b43763-afaf-4f36-a8ef-c03a3cd70eb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:44.397448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:44.399595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 171/350] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:44.401536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b43763-afaf-4f36-a8ef-c03a3cd70eb3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:44.431703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b805-9bec-4074-b3b2-2033ae25ee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:44.432346 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b805-9bec-4074-b3b2-2033ae25ee32 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.432632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b805-9bec-4074-b3b2-2033ae25ee32 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.432905 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b805-9bec-4074-b3b2-2033ae25ee32 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.434089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005b805-9bec-4074-b3b2-2033ae25ee32 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:44.435530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005b805-9bec-4074-b3b2-2033ae25ee32 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:44.437648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 180/351] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:44 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:44.438487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b805-9bec-4074-b3b2-2033ae25ee32 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:44.529679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b43763-afaf-4f36-a8ef-c03a3cd70eb3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:44.531233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:44.569800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b805-9bec-4074-b3b2-2033ae25ee32 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:44.571356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f498a8-370c-4648-9a22-b68e2b8c84cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:44.571992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f498a8-370c-4648-9a22-b68e2b8c84cd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.572272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f498a8-370c-4648-9a22-b68e2b8c84cd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.572542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f498a8-370c-4648-9a22-b68e2b8c84cd None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.573737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f498a8-370c-4648-9a22-b68e2b8c84cd None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:44.575461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f498a8-370c-4648-9a22-b68e2b8c84cd None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:44.577106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 172/352] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:44 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:44.577934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f498a8-370c-4648-9a22-b68e2b8c84cd None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:44.613253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:44.624537 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097dc73-8d1d-4a21-a598-d402a7f71abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:44.625156 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097dc73-8d1d-4a21-a598-d402a7f71abb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.628524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097dc73-8d1d-4a21-a598-d402a7f71abb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.628524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097dc73-8d1d-4a21-a598-d402a7f71abb None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.628524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097dc73-8d1d-4a21-a598-d402a7f71abb None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:44.628524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097dc73-8d1d-4a21-a598-d402a7f71abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:44.628933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097dc73-8d1d-4a21-a598-d402a7f71abb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:44.644622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 181/353] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:44.646008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097dc73-8d1d-4a21-a598-d402a7f71abb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:44.719931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f498a8-370c-4648-9a22-b68e2b8c84cd None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:44.758551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097dc73-8d1d-4a21-a598-d402a7f71abb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:44.764281 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346fd8-bab2-4fcd-81b3-9172532bd7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:44.765116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346fd8-bab2-4fcd-81b3-9172532bd7ef None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.765569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346fd8-bab2-4fcd-81b3-9172532bd7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.765995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346fd8-bab2-4fcd-81b3-9172532bd7ef None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.767247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38346fd8-bab2-4fcd-81b3-9172532bd7ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:44.767713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38346fd8-bab2-4fcd-81b3-9172532bd7ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:44.770202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38346fd8-bab2-4fcd-81b3-9172532bd7ef None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:44.782882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 173/354] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:44.784037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733bd64-aa88-455f-bc3d-21015ddf2bbc None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.784374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733bd64-aa88-455f-bc3d-21015ddf2bbc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.784671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733bd64-aa88-455f-bc3d-21015ddf2bbc None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.791602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 174/355] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:44.792623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733bd64-aa88-455f-bc3d-21015ddf2bbc None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:44.811983 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3fd1a-32f3-4b24-a486-691057b66f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:44.812973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3fd1a-32f3-4b24-a486-691057b66f97 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.813248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3fd1a-32f3-4b24-a486-691057b66f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.813724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3fd1a-32f3-4b24-a486-691057b66f97 None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.821024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3fd1a-32f3-4b24-a486-691057b66f97 None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:44.822707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3fd1a-32f3-4b24-a486-691057b66f97 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:44.825097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 182/356] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:44 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:44.826097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3fd1a-32f3-4b24-a486-691057b66f97 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:44.831031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:44.901670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:44.913708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733bd64-aa88-455f-bc3d-21015ddf2bbc None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:44.953239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ed204-aff3-47c5-be37-21d1f9b788cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:44.953914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ed204-aff3-47c5-be37-21d1f9b788cd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.954188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ed204-aff3-47c5-be37-21d1f9b788cd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.954460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ed204-aff3-47c5-be37-21d1f9b788cd None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.955616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ed204-aff3-47c5-be37-21d1f9b788cd None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:44.957867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ed204-aff3-47c5-be37-21d1f9b788cd None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:44.964641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 175/357] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:44 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:44.964641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ed204-aff3-47c5-be37-21d1f9b788cd None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:44.964937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3fd1a-32f3-4b24-a486-691057b66f97 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:45.007892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1ab20-d389-4990-8218-d3693342ac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:45.008507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1ab20-d389-4990-8218-d3693342ac6c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.008930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1ab20-d389-4990-8218-d3693342ac6c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.011977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1ab20-d389-4990-8218-d3693342ac6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.082705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:45.084806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ed204-aff3-47c5-be37-21d1f9b788cd None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:45.091279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1ab20-d389-4990-8218-d3693342ac6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:45.093197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1ab20-d389-4990-8218-d3693342ac6c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:45.127052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a70fcd-3c6f-49b8-b0ff-4c103bea57db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:45.127685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a70fcd-3c6f-49b8-b0ff-4c103bea57db None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.128594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a70fcd-3c6f-49b8-b0ff-4c103bea57db None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.128860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a70fcd-3c6f-49b8-b0ff-4c103bea57db None admin] PATH_INFO: `/v3/users/25cfddf078b84199982756003b82dc05` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.130236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a70fcd-3c6f-49b8-b0ff-4c103bea57db None admin] RBAC: Authorizing `identity:get_user(user_id=25cfddf078b84199982756003b82dc05)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:45.131686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a70fcd-3c6f-49b8-b0ff-4c103bea57db None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:45.133291 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 176/358] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:44 2022] GET /identity/v3/users/25cfddf078b84199982756003b82dc05 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:45.134698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f245e5-2f1c-4b6f-95de-c21d9354f3c6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.134998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f245e5-2f1c-4b6f-95de-c21d9354f3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.135254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f245e5-2f1c-4b6f-95de-c21d9354f3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.157572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-38f245e5-2f1c-4b6f-95de-c21d9354f3c6 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:45.158764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-38f245e5-2f1c-4b6f-95de-c21d9354f3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:45.166525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:45.221096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 183/359] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:45.222786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1ab20-d389-4990-8218-d3693342ac6c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:45.241045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 177/360] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:45.246399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b80474-7f29-4342-85f4-512e498d72d8 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.246691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b80474-7f29-4342-85f4-512e498d72d8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.246964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b80474-7f29-4342-85f4-512e498d72d8 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.248415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 178/361] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:45.249024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b80474-7f29-4342-85f4-512e498d72d8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:45.358727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1ab20-d389-4990-8218-d3693342ac6c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:45.380274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:45.384712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b80474-7f29-4342-85f4-512e498d72d8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:45.398863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde12e4-13fd-44dd-8cf6-4892268ac19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:45.399482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde12e4-13fd-44dd-8cf6-4892268ac19a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.399758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde12e4-13fd-44dd-8cf6-4892268ac19a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.400048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde12e4-13fd-44dd-8cf6-4892268ac19a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.400761 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde12e4-13fd-44dd-8cf6-4892268ac19a None admin] RBAC: Adding query filter params (group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, scope.project.id=812094fb94404914876df4528f073add) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:45.401067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde12e4-13fd-44dd-8cf6-4892268ac19a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:45.402543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde12e4-13fd-44dd-8cf6-4892268ac19a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:45.408973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 184/362] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 1 05:12:45 2022] GET /identity/v3/role_assignments?group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8&scope.project.id=812094fb94404914876df4528f073add&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:45.411877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde12e4-13fd-44dd-8cf6-4892268ac19a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:45.423417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e52377-8738-4a2b-96bf-6edfbbe3a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:45.424354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e52377-8738-4a2b-96bf-6edfbbe3a773 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.424652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e52377-8738-4a2b-96bf-6edfbbe3a773 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.425855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e52377-8738-4a2b-96bf-6edfbbe3a773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.426943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e52377-8738-4a2b-96bf-6edfbbe3a773 None admin] RBAC: Adding query filter params (group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role.id=acfe3eeb91964f62abb1d8a84c36dccc, scope.project.id=812094fb94404914876df4528f073add) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:45.427255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e52377-8738-4a2b-96bf-6edfbbe3a773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:45.429404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e52377-8738-4a2b-96bf-6edfbbe3a773 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:45.441662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 179/363] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 1 05:12:45 2022] GET /identity/v3/role_assignments?group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8&scope.project.id=812094fb94404914876df4528f073add&role.id=acfe3eeb91964f62abb1d8a84c36dccc => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:45.442362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:45.443018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e52377-8738-4a2b-96bf-6edfbbe3a773 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:45.591252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde12e4-13fd-44dd-8cf6-4892268ac19a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:45.619449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e52377-8738-4a2b-96bf-6edfbbe3a773 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:45.636594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835f6bf-3e17-40ca-971f-2616629ba172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:45.637218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835f6bf-3e17-40ca-971f-2616629ba172 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.637612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835f6bf-3e17-40ca-971f-2616629ba172 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.637808 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835f6bf-3e17-40ca-971f-2616629ba172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.638545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835f6bf-3e17-40ca-971f-2616629ba172 None admin] RBAC: Adding query filter params (role.id=d529d6d058424562a72162b0c257851f, user.id=25cfddf078b84199982756003b82dc05, scope.system=all) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:45.638847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835f6bf-3e17-40ca-971f-2616629ba172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:45.640263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835f6bf-3e17-40ca-971f-2616629ba172 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:45.654628 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 185/364] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 1 05:12:45 2022] GET /identity/v3/role_assignments?user.id=25cfddf078b84199982756003b82dc05&scope.system=all&role.id=d529d6d058424562a72162b0c257851f => generated 510 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:45.656381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d05ea-fbe9-4c5b-bbc4-03af7c38985f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.656978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d05ea-fbe9-4c5b-bbc4-03af7c38985f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.657448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d05ea-fbe9-4c5b-bbc4-03af7c38985f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.669393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b451f3c-c324-4b59-bd69-85d9c5bd61aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:45.670192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b451f3c-c324-4b59-bd69-85d9c5bd61aa None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.670477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b451f3c-c324-4b59-bd69-85d9c5bd61aa None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.670753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b451f3c-c324-4b59-bd69-85d9c5bd61aa None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.671922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b451f3c-c324-4b59-bd69-85d9c5bd61aa None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:45.673378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b451f3c-c324-4b59-bd69-85d9c5bd61aa None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:45.675456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 180/365] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:45 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:45.676286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b451f3c-c324-4b59-bd69-85d9c5bd61aa None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:45.687488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dd8d05ea-fbe9-4c5b-bbc4-03af7c38985f None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:45.691223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dd8d05ea-fbe9-4c5b-bbc4-03af7c38985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:45.721365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:45.829628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b451f3c-c324-4b59-bd69-85d9c5bd61aa None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:45.870135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 186/366] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:45.871039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d05ea-fbe9-4c5b-bbc4-03af7c38985f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:45.871601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d5660-cc2b-429f-bf63-60088aea076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:45.872278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d5660-cc2b-429f-bf63-60088aea076f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.872511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d5660-cc2b-429f-bf63-60088aea076f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.872776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d5660-cc2b-429f-bf63-60088aea076f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.891723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d5660-cc2b-429f-bf63-60088aea076f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:45.894301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-a95d5660-cc2b-429f-bf63-60088aea076f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:45.898219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:45.899130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:45.899939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:45.909432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 181/367] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:45.910573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d5660-cc2b-429f-bf63-60088aea076f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:46.028335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d05ea-fbe9-4c5b-bbc4-03af7c38985f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:46.059601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:46.080161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d5660-cc2b-429f-bf63-60088aea076f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:46.082525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ccbdf-bc9b-4611-9f0c-52e255109145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:46.083146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ccbdf-bc9b-4611-9f0c-52e255109145 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:46.083425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ccbdf-bc9b-4611-9f0c-52e255109145 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:46.083698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ccbdf-bc9b-4611-9f0c-52e255109145 None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:46.086351 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ccbdf-bc9b-4611-9f0c-52e255109145 None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:46.087799 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ccbdf-bc9b-4611-9f0c-52e255109145 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:46.094537 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 187/368] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:45 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:46.126904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397345c8-8ed4-4d59-976f-4c023183068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:46.127436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397345c8-8ed4-4d59-976f-4c023183068c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:46.127712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397345c8-8ed4-4d59-976f-4c023183068c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:46.128519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397345c8-8ed4-4d59-976f-4c023183068c None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:46.129193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397345c8-8ed4-4d59-976f-4c023183068c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:46.129514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397345c8-8ed4-4d59-976f-4c023183068c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:46.131117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397345c8-8ed4-4d59-976f-4c023183068c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:46.134719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ccbdf-bc9b-4611-9f0c-52e255109145 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:46.156441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 182/369] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:46.161843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397345c8-8ed4-4d59-976f-4c023183068c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:46.257451 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:46.294832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ccbdf-bc9b-4611-9f0c-52e255109145 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:46.320113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397345c8-8ed4-4d59-976f-4c023183068c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:46.335004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584319f-b2bd-488b-9a29-768e61844bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:46.335732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584319f-b2bd-488b-9a29-768e61844bcf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:46.336004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584319f-b2bd-488b-9a29-768e61844bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:46.336383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584319f-b2bd-488b-9a29-768e61844bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:46.337121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584319f-b2bd-488b-9a29-768e61844bcf None admin] RBAC: Adding query filter params (group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, scope.project.id=d61e52cca9584059bf484cabd52d2808) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:46.337422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584319f-b2bd-488b-9a29-768e61844bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:46.338970 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584319f-b2bd-488b-9a29-768e61844bcf None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:46.345105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 188/370] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 1 05:12:46 2022] GET /identity/v3/role_assignments?group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:46.349208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584319f-b2bd-488b-9a29-768e61844bcf None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:46.377700 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64385c43-0e40-458a-9ccc-4174234d7e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:46.378281 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64385c43-0e40-458a-9ccc-4174234d7e77 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:46.378555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64385c43-0e40-458a-9ccc-4174234d7e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:46.378820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64385c43-0e40-458a-9ccc-4174234d7e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:46.421855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:46.501035 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64385c43-0e40-458a-9ccc-4174234d7e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:46.502550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64385c43-0e40-458a-9ccc-4174234d7e77 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:46.554918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584319f-b2bd-488b-9a29-768e61844bcf None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:46.597179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86ad09-bb17-432f-b560-1a58bb8312e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:46.598214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86ad09-bb17-432f-b560-1a58bb8312e0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:46.598714 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86ad09-bb17-432f-b560-1a58bb8312e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:46.599236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86ad09-bb17-432f-b560-1a58bb8312e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:46.613697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:46.618704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86ad09-bb17-432f-b560-1a58bb8312e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-aa86ad09-bb17-432f-b560-1a58bb8312e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:46.624639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:46.631952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:46.632786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 189/371] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:46.637034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86ad09-bb17-432f-b560-1a58bb8312e0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:46.656568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 183/372] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:12:46.663534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64385c43-0e40-458a-9ccc-4174234d7e77 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:46.767890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:46.819659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64385c43-0e40-458a-9ccc-4174234d7e77 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:46.854082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86ad09-bb17-432f-b560-1a58bb8312e0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:46.861153 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101ec9c-9f23-441e-b29b-401dc9d549fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:46.861861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101ec9c-9f23-441e-b29b-401dc9d549fc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:46.862192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101ec9c-9f23-441e-b29b-401dc9d549fc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:46.862669 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101ec9c-9f23-441e-b29b-401dc9d549fc None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:46.863823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101ec9c-9f23-441e-b29b-401dc9d549fc None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:46.865295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101ec9c-9f23-441e-b29b-401dc9d549fc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:46.867250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 184/373] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:46 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:46.872582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101ec9c-9f23-441e-b29b-401dc9d549fc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:46.942447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24744eb-2fcc-424d-a212-56121ff11a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:46.942998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24744eb-2fcc-424d-a212-56121ff11a2b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:46.943283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24744eb-2fcc-424d-a212-56121ff11a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:46.943561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24744eb-2fcc-424d-a212-56121ff11a2b None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:46.944240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24744eb-2fcc-424d-a212-56121ff11a2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:46.944548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24744eb-2fcc-424d-a212-56121ff11a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:46.945965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24744eb-2fcc-424d-a212-56121ff11a2b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:46.966063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 190/374] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:46.972069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24744eb-2fcc-424d-a212-56121ff11a2b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:46.983861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101ec9c-9f23-441e-b29b-401dc9d549fc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:47.013538 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:47.031887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270dfd2-fb23-4c37-8ba0-3525492e8a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:47.032528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270dfd2-fb23-4c37-8ba0-3525492e8a7c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.032806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270dfd2-fb23-4c37-8ba0-3525492e8a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.033080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270dfd2-fb23-4c37-8ba0-3525492e8a7c None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.039293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270dfd2-fb23-4c37-8ba0-3525492e8a7c None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:47.040778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270dfd2-fb23-4c37-8ba0-3525492e8a7c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:47.047796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 185/375] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:46 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:47.056012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270dfd2-fb23-4c37-8ba0-3525492e8a7c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:47.067174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:47.111315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24744eb-2fcc-424d-a212-56121ff11a2b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:47.149336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29713d-b6c9-421b-bb05-121f403ecdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:47.149989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29713d-b6c9-421b-bb05-121f403ecdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.150371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29713d-b6c9-421b-bb05-121f403ecdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.150843 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29713d-b6c9-421b-bb05-121f403ecdc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.215849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270dfd2-fb23-4c37-8ba0-3525492e8a7c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:47.227366 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29713d-b6c9-421b-bb05-121f403ecdc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:47.228914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29713d-b6c9-421b-bb05-121f403ecdc8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:47.264571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec4ce7-672b-49cb-a2e1-cf26646aaed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:47.265190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec4ce7-672b-49cb-a2e1-cf26646aaed9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.265482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec4ce7-672b-49cb-a2e1-cf26646aaed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.265789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec4ce7-672b-49cb-a2e1-cf26646aaed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.266521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec4ce7-672b-49cb-a2e1-cf26646aaed9 None admin] RBAC: Adding query filter params (group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role.id=acfe3eeb91964f62abb1d8a84c36dccc, scope.project.id=d61e52cca9584059bf484cabd52d2808) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:47.266833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec4ce7-672b-49cb-a2e1-cf26646aaed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:47.268302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec4ce7-672b-49cb-a2e1-cf26646aaed9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:47.277381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 186/376] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 1 05:12:47 2022] GET /identity/v3/role_assignments?group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=acfe3eeb91964f62abb1d8a84c36dccc => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:47.297472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:47.345146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 191/377] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:47.351436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec4ce7-672b-49cb-a2e1-cf26646aaed9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:47.432440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:47.490193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec4ce7-672b-49cb-a2e1-cf26646aaed9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:47.529751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535408b-f138-4479-b0e1-51e6243a5292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:47.530362 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535408b-f138-4479-b0e1-51e6243a5292 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.530645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535408b-f138-4479-b0e1-51e6243a5292 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.530921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535408b-f138-4479-b0e1-51e6243a5292 None admin] PATH_INFO: `/v3/projects/61f9dee9572644e580e35da89b6265e9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.532059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535408b-f138-4479-b0e1-51e6243a5292 None admin] RBAC: Authorizing `identity:get_project(project_id=61f9dee9572644e580e35da89b6265e9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:47.533535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535408b-f138-4479-b0e1-51e6243a5292 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:47.535567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 187/378] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:47 2022] GET /identity/v3/projects/61f9dee9572644e580e35da89b6265e9 => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:47.540667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29713d-b6c9-421b-bb05-121f403ecdc8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:47.659243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29713d-b6c9-421b-bb05-121f403ecdc8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:47.701302 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afba340-8563-41a6-baa7-f9815e672b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:47.701982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afba340-8563-41a6-baa7-f9815e672b02 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.702267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afba340-8563-41a6-baa7-f9815e672b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.702532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afba340-8563-41a6-baa7-f9815e672b02 None admin] PATH_INFO: `/v3/groups/1a8a7be2623642c19a79fdd0965fdf22` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.709189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afba340-8563-41a6-baa7-f9815e672b02 None admin] RBAC: Authorizing `identity:get_group(group_id=1a8a7be2623642c19a79fdd0965fdf22)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:47.710778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afba340-8563-41a6-baa7-f9815e672b02 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:47.712443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 192/379] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:47 2022] GET /identity/v3/groups/1a8a7be2623642c19a79fdd0965fdf22 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:47.719698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535408b-f138-4479-b0e1-51e6243a5292 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:47.848695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535408b-f138-4479-b0e1-51e6243a5292 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:47.870160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:47.884580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d4306-f6a0-4c55-a333-5f8d2fd5f9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:47.896465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d4306-f6a0-4c55-a333-5f8d2fd5f9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.896465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d4306-f6a0-4c55-a333-5f8d2fd5f9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.896465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d4306-f6a0-4c55-a333-5f8d2fd5f9c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.896465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d4306-f6a0-4c55-a333-5f8d2fd5f9c2 None admin] RBAC: Adding query filter params (group.id=1a8a7be2623642c19a79fdd0965fdf22, role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=61f9dee9572644e580e35da89b6265e9) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:47.896465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d4306-f6a0-4c55-a333-5f8d2fd5f9c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:47.896465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d4306-f6a0-4c55-a333-5f8d2fd5f9c2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:47.896876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 188/380] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 1 05:12:47 2022] GET /identity/v3/role_assignments?group.id=1a8a7be2623642c19a79fdd0965fdf22&scope.project.id=61f9dee9572644e580e35da89b6265e9&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:47.970317 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b183-6b16-4ab6-a021-21118943ee86 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.970695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b183-6b16-4ab6-a021-21118943ee86 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.971051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b183-6b16-4ab6-a021-21118943ee86 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.973034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 193/381] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:47.988105 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6a3fb-9582-4027-9671-15bd367a5eef None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.988676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6a3fb-9582-4027-9671-15bd367a5eef None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.989132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6a3fb-9582-4027-9671-15bd367a5eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:48.014967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c5e6a3fb-9582-4027-9671-15bd367a5eef None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:48.016577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c5e6a3fb-9582-4027-9671-15bd367a5eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:48.099535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 189/382] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:48.383743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40b183-6b16-4ab6-a021-21118943ee86 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:48.400458 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2da2d3-fff5-4190-80fb-c49470b75b80 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:48.400737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2da2d3-fff5-4190-80fb-c49470b75b80 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:48.401015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2da2d3-fff5-4190-80fb-c49470b75b80 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:48.402640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 190/383] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:48.407866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9017-a6b5-4a39-ad56-0b9b5b4e027d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:48.408132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9017-a6b5-4a39-ad56-0b9b5b4e027d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:48.408403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9017-a6b5-4a39-ad56-0b9b5b4e027d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:48.415877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:48.444654 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-856c9017-a6b5-4a39-ad56-0b9b5b4e027d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:48.448545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-856c9017-a6b5-4a39-ad56-0b9b5b4e027d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:48.548391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40b183-6b16-4ab6-a021-21118943ee86 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:48.591033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec178027-bea5-4d1a-ac2d-a55497aed4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:48.592306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec178027-bea5-4d1a-ac2d-a55497aed4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:48.592790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec178027-bea5-4d1a-ac2d-a55497aed4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:48.593384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec178027-bea5-4d1a-ac2d-a55497aed4c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:48.605335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec178027-bea5-4d1a-ac2d-a55497aed4c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:48.614345 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 191/384] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ec178027-bea5-4d1a-ac2d-a55497aed4c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:48.614830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:48.615730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:48.616547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:48.620128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 194/385] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:48.629685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9017-a6b5-4a39-ad56-0b9b5b4e027d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:48.713229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:48.732602 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858aafb-7950-4c4d-bef6-0982d9e64450 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:48.733055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858aafb-7950-4c4d-bef6-0982d9e64450 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:48.733332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858aafb-7950-4c4d-bef6-0982d9e64450 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:48.734993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 195/386] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:48.742203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc2e75-6e5a-4e72-8ae5-f08e3b80fa09 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:48.742203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc2e75-6e5a-4e72-8ae5-f08e3b80fa09 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:48.742203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc2e75-6e5a-4e72-8ae5-f08e3b80fa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:48.777227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-99fc2e75-6e5a-4e72-8ae5-f08e3b80fa09 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:48.778554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-99fc2e75-6e5a-4e72-8ae5-f08e3b80fa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:48.803676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9017-a6b5-4a39-ad56-0b9b5b4e027d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:48.858917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 196/387] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:48.867164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db71256-13e1-44c8-8271-73187a11d546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:48.867983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db71256-13e1-44c8-8271-73187a11d546 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:48.868409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db71256-13e1-44c8-8271-73187a11d546 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:48.868833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db71256-13e1-44c8-8271-73187a11d546 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:48.869695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db71256-13e1-44c8-8271-73187a11d546 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:48.870144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db71256-13e1-44c8-8271-73187a11d546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:48.871691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db71256-13e1-44c8-8271-73187a11d546 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:48.888939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 192/388] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:48.895051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc2e75-6e5a-4e72-8ae5-f08e3b80fa09 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:48.923271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:49.031311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc2e75-6e5a-4e72-8ae5-f08e3b80fa09 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:49.070994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe10123-af89-48f7-928e-88fceb2559ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:49.071635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe10123-af89-48f7-928e-88fceb2559ab None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.071918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe10123-af89-48f7-928e-88fceb2559ab None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.072196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe10123-af89-48f7-928e-88fceb2559ab None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.073306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe10123-af89-48f7-928e-88fceb2559ab None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:49.075127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe10123-af89-48f7-928e-88fceb2559ab None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:49.080011 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 197/389] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:48 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:49.082772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db71256-13e1-44c8-8271-73187a11d546 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:49.171392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe10123-af89-48f7-928e-88fceb2559ab None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:49.237716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db71256-13e1-44c8-8271-73187a11d546 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:49.252412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:49.274992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:49.286335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e4e76-01aa-4802-a7a9-916ff66bc786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:49.286901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e4e76-01aa-4802-a7a9-916ff66bc786 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.287319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e4e76-01aa-4802-a7a9-916ff66bc786 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.287601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e4e76-01aa-4802-a7a9-916ff66bc786 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.289310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e4e76-01aa-4802-a7a9-916ff66bc786 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:49.290966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e4e76-01aa-4802-a7a9-916ff66bc786 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:49.293503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 193/390] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:49 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:49.294238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe10123-af89-48f7-928e-88fceb2559ab None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:49.294378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e4e76-01aa-4802-a7a9-916ff66bc786 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:49.340104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319fed2-33eb-4ae8-90e0-7d47e5766f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:49.341135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fed2-33eb-4ae8-90e0-7d47e5766f43 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.341536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fed2-33eb-4ae8-90e0-7d47e5766f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.342863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fed2-33eb-4ae8-90e0-7d47e5766f43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.355280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319fed2-33eb-4ae8-90e0-7d47e5766f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0319fed2-33eb-4ae8-90e0-7d47e5766f43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:49.361717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:49.363347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:49.364163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 198/391] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:49.364840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319fed2-33eb-4ae8-90e0-7d47e5766f43 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:49.422313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e4e76-01aa-4802-a7a9-916ff66bc786 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:49.461257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d002b-d31a-4b47-b4e2-2f405a180de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:49.461926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d002b-d31a-4b47-b4e2-2f405a180de4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.462183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d002b-d31a-4b47-b4e2-2f405a180de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.462454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d002b-d31a-4b47-b4e2-2f405a180de4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.474217 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319fed2-33eb-4ae8-90e0-7d47e5766f43 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:49.475748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d002b-d31a-4b47-b4e2-2f405a180de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-1b5d002b-d31a-4b47-b4e2-2f405a180de4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:49.485612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:49.486584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:49.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:49.493816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 194/392] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:49.494700 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d002b-d31a-4b47-b4e2-2f405a180de4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:49.515330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65859ac-19b5-45b2-a73c-98c49a3da6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:49.516303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65859ac-19b5-45b2-a73c-98c49a3da6ce None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.516795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65859ac-19b5-45b2-a73c-98c49a3da6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.517291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65859ac-19b5-45b2-a73c-98c49a3da6ce None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.519619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65859ac-19b5-45b2-a73c-98c49a3da6ce None admin] RBAC: Authorizing `identity:create_grant(project_id=812094fb94404914876df4528f073add, group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:49.521256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65859ac-19b5-45b2-a73c-98c49a3da6ce None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:49.533829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:49.609178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d002b-d31a-4b47-b4e2-2f405a180de4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:49.619910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 199/393] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 1 05:12:49 2022] PUT /identity/v3/projects/812094fb94404914876df4528f073add/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:49.621270 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65859ac-19b5-45b2-a73c-98c49a3da6ce None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:49.640192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:49.688772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140a43b-4f07-42a2-bbd8-d6ee270a541a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:49.689398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140a43b-4f07-42a2-bbd8-d6ee270a541a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.689694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140a43b-4f07-42a2-bbd8-d6ee270a541a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.690003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140a43b-4f07-42a2-bbd8-d6ee270a541a None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.693926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140a43b-4f07-42a2-bbd8-d6ee270a541a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:49.694270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140a43b-4f07-42a2-bbd8-d6ee270a541a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:49.695720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140a43b-4f07-42a2-bbd8-d6ee270a541a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:49.727836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 195/394] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:49.729943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5118-77b5-4b00-bfd2-fcd07b210f9c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.730230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5118-77b5-4b00-bfd2-fcd07b210f9c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.730504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5118-77b5-4b00-bfd2-fcd07b210f9c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.732098 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 196/395] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:49.738003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5118-77b5-4b00-bfd2-fcd07b210f9c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:49.819174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65859ac-19b5-45b2-a73c-98c49a3da6ce None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:49.864673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a33a4-8cbb-4326-be32-7106a036140c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:49.865596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a33a4-8cbb-4326-be32-7106a036140c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.866014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a33a4-8cbb-4326-be32-7106a036140c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.866446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a33a4-8cbb-4326-be32-7106a036140c None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.867261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a33a4-8cbb-4326-be32-7106a036140c None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:49.867747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a33a4-8cbb-4326-be32-7106a036140c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:49.869312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a33a4-8cbb-4326-be32-7106a036140c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:49.869672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:49.877048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5118-77b5-4b00-bfd2-fcd07b210f9c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:49.879654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 200/396] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:49.885376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a36a81-f214-47bd-bfa0-836b1a5dc31a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.885937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a36a81-f214-47bd-bfa0-836b1a5dc31a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.886357 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a36a81-f214-47bd-bfa0-836b1a5dc31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.910608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-50a36a81-f214-47bd-bfa0-836b1a5dc31a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:49.912692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-50a36a81-f214-47bd-bfa0-836b1a5dc31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:49.916444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a339b55-198e-45a0-89de-99c70939d3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:49.916721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a339b55-198e-45a0-89de-99c70939d3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.917048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a339b55-198e-45a0-89de-99c70939d3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.917325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a339b55-198e-45a0-89de-99c70939d3d3 None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.918519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a339b55-198e-45a0-89de-99c70939d3d3 None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:49.920035 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a339b55-198e-45a0-89de-99c70939d3d3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:49.922062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 197/397] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:49 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:49.923083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a339b55-198e-45a0-89de-99c70939d3d3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:49.993211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:50.005581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 201/398] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:50.006554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a36a81-f214-47bd-bfa0-836b1a5dc31a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:50.046935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a339b55-198e-45a0-89de-99c70939d3d3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:50.089389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b1101-8622-4244-b8a0-ef4bc5b6e336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:50.090283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b1101-8622-4244-b8a0-ef4bc5b6e336 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:50.090796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b1101-8622-4244-b8a0-ef4bc5b6e336 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:50.091203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b1101-8622-4244-b8a0-ef4bc5b6e336 None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:50.092587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b1101-8622-4244-b8a0-ef4bc5b6e336 None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:50.094348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b1101-8622-4244-b8a0-ef4bc5b6e336 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:50.096120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 198/399] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:49 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:50.103038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b1101-8622-4244-b8a0-ef4bc5b6e336 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:50.145094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:50.227991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a36a81-f214-47bd-bfa0-836b1a5dc31a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:50.262693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b1101-8622-4244-b8a0-ef4bc5b6e336 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:50.270806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f6033-1f47-4f88-8e63-4ecee791ba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:50.271628 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f6033-1f47-4f88-8e63-4ecee791ba59 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:50.272006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f6033-1f47-4f88-8e63-4ecee791ba59 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:50.272415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f6033-1f47-4f88-8e63-4ecee791ba59 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:50.273688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f6033-1f47-4f88-8e63-4ecee791ba59 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:50.275236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f6033-1f47-4f88-8e63-4ecee791ba59 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:50.281482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 202/400] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:50 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:50.286835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f6033-1f47-4f88-8e63-4ecee791ba59 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:50.329355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af0ba4-80b5-4c1a-b5c1-91a2368290c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:50.330024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af0ba4-80b5-4c1a-b5c1-91a2368290c7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:50.330299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af0ba4-80b5-4c1a-b5c1-91a2368290c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:50.330564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af0ba4-80b5-4c1a-b5c1-91a2368290c7 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:50.332211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af0ba4-80b5-4c1a-b5c1-91a2368290c7 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:50.333680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af0ba4-80b5-4c1a-b5c1-91a2368290c7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:50.335699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 199/401] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:50 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:50.336888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f400de-80a5-4b96-a39d-265cb9f6820f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:50.337161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f400de-80a5-4b96-a39d-265cb9f6820f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:50.337569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f400de-80a5-4b96-a39d-265cb9f6820f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:50.338744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 200/402] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:50.343405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f400de-80a5-4b96-a39d-265cb9f6820f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:50.362789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:50.401671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f6033-1f47-4f88-8e63-4ecee791ba59 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:50.442236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164456d-b4a4-458d-ba5b-c9177035ec08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:50.442995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164456d-b4a4-458d-ba5b-c9177035ec08 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:50.443532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164456d-b4a4-458d-ba5b-c9177035ec08 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:50.443728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164456d-b4a4-458d-ba5b-c9177035ec08 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8/roles/acfe3eeb91964f62abb1d8a84c36dccc` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:50.446020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164456d-b4a4-458d-ba5b-c9177035ec08 None admin] RBAC: Authorizing `identity:create_grant(project_id=812094fb94404914876df4528f073add, group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role_id=acfe3eeb91964f62abb1d8a84c36dccc)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:50.447528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164456d-b4a4-458d-ba5b-c9177035ec08 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:50.449506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:50.453937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f400de-80a5-4b96-a39d-265cb9f6820f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:50.489473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b85d7-ea58-443c-8f0d-1807b9706278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:50.490132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b85d7-ea58-443c-8f0d-1807b9706278 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:50.490393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b85d7-ea58-443c-8f0d-1807b9706278 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:50.490664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b85d7-ea58-443c-8f0d-1807b9706278 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:50.492534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b85d7-ea58-443c-8f0d-1807b9706278 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61e52cca9584059bf484cabd52d2808, group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:50.494071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b85d7-ea58-443c-8f0d-1807b9706278 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:50.528995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 203/403] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 1 05:12:50 2022] PUT /identity/v3/projects/812094fb94404914876df4528f073add/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8/roles/acfe3eeb91964f62abb1d8a84c36dccc => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:12:50.534005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672ea56-9d95-4436-80f0-fd7cd14f30a2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:50.534005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672ea56-9d95-4436-80f0-fd7cd14f30a2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:50.534005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672ea56-9d95-4436-80f0-fd7cd14f30a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:50.550531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1672ea56-9d95-4436-80f0-fd7cd14f30a2 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:50.552115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1672ea56-9d95-4436-80f0-fd7cd14f30a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:50.621990 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 201/404] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 1 05:12:50 2022] PUT /identity/v3/projects/d61e52cca9584059bf484cabd52d2808/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:50.623365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b85d7-ea58-443c-8f0d-1807b9706278 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:50.663368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 204/405] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:50.772785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b85d7-ea58-443c-8f0d-1807b9706278 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:50.813192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2f19b-fd57-4526-a741-53cf4126646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:50.813867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2f19b-fd57-4526-a741-53cf4126646d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:50.814146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2f19b-fd57-4526-a741-53cf4126646d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:50.814415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2f19b-fd57-4526-a741-53cf4126646d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:50.826444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2f19b-fd57-4526-a741-53cf4126646d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cad2f19b-fd57-4526-a741-53cf4126646d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:50.832250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:50.833128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:50.834373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 202/406] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:50.841064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672ea56-9d95-4436-80f0-fd7cd14f30a2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:50.912402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:50.976659 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672ea56-9d95-4436-80f0-fd7cd14f30a2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:51.012642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f728c2-ccba-43fb-8e36-55bbfae09dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:51.013253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f728c2-ccba-43fb-8e36-55bbfae09dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:51.013560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f728c2-ccba-43fb-8e36-55bbfae09dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:51.013835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f728c2-ccba-43fb-8e36-55bbfae09dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:51.014500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f728c2-ccba-43fb-8e36-55bbfae09dd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:51.014804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f728c2-ccba-43fb-8e36-55bbfae09dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:51.016176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f728c2-ccba-43fb-8e36-55bbfae09dd6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:51.020389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2f19b-fd57-4526-a741-53cf4126646d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:51.025778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 205/407] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:51.036240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f728c2-ccba-43fb-8e36-55bbfae09dd6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:51.069336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:51.208180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2f19b-fd57-4526-a741-53cf4126646d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:51.249612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f728c2-ccba-43fb-8e36-55bbfae09dd6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:51.261556 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2937c4-bb60-4316-aa01-cd77e73ab595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:51.262472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2937c4-bb60-4316-aa01-cd77e73ab595 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:51.262929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2937c4-bb60-4316-aa01-cd77e73ab595 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:51.263375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2937c4-bb60-4316-aa01-cd77e73ab595 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:51.283208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:51.286146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2937c4-bb60-4316-aa01-cd77e73ab595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-dc2937c4-bb60-4316-aa01-cd77e73ab595 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:51.296597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:51.307925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 203/408] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:51.308791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2937c4-bb60-4316-aa01-cd77e73ab595 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:51.322265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf73ecb-0c13-4e08-8e27-9c637154c07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:51.322917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf73ecb-0c13-4e08-8e27-9c637154c07e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:51.323200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf73ecb-0c13-4e08-8e27-9c637154c07e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:51.323475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf73ecb-0c13-4e08-8e27-9c637154c07e None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:51.325349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf73ecb-0c13-4e08-8e27-9c637154c07e None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:51.327457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf73ecb-0c13-4e08-8e27-9c637154c07e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:51.330741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 206/409] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:51 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:51.337075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf73ecb-0c13-4e08-8e27-9c637154c07e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:51.451475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:51.486747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2937c4-bb60-4316-aa01-cd77e73ab595 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:51.523306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf73ecb-0c13-4e08-8e27-9c637154c07e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:51.545402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ec43-c3ad-4abf-b1b5-10bab94a34d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:51.546033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ec43-c3ad-4abf-b1b5-10bab94a34d7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:51.546313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ec43-c3ad-4abf-b1b5-10bab94a34d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:51.546582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ec43-c3ad-4abf-b1b5-10bab94a34d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:51.547244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141ec43-c3ad-4abf-b1b5-10bab94a34d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:51.547545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141ec43-c3ad-4abf-b1b5-10bab94a34d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:51.548936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141ec43-c3ad-4abf-b1b5-10bab94a34d7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:51.569094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 204/410] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:51.575037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ec43-c3ad-4abf-b1b5-10bab94a34d7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:51.594275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98fed6-e7ed-4eb8-b42a-de9fcfb67352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:51.594921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98fed6-e7ed-4eb8-b42a-de9fcfb67352 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:51.595201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98fed6-e7ed-4eb8-b42a-de9fcfb67352 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:51.595482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98fed6-e7ed-4eb8-b42a-de9fcfb67352 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:51.596776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98fed6-e7ed-4eb8-b42a-de9fcfb67352 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:51.598245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98fed6-e7ed-4eb8-b42a-de9fcfb67352 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:51.599876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 207/411] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:51 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:51.605938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98fed6-e7ed-4eb8-b42a-de9fcfb67352 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:51.699764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:51.774507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98fed6-e7ed-4eb8-b42a-de9fcfb67352 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:51.795110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ec43-c3ad-4abf-b1b5-10bab94a34d7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:51.818103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1088006-4d2b-4de5-968e-9e6fb8a890c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:51.818875 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1088006-4d2b-4de5-968e-9e6fb8a890c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:51.819619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1088006-4d2b-4de5-968e-9e6fb8a890c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:51.819899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1088006-4d2b-4de5-968e-9e6fb8a890c4 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8/roles/acfe3eeb91964f62abb1d8a84c36dccc` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:51.822302 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1088006-4d2b-4de5-968e-9e6fb8a890c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d61e52cca9584059bf484cabd52d2808, group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role_id=acfe3eeb91964f62abb1d8a84c36dccc)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:51.823752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1088006-4d2b-4de5-968e-9e6fb8a890c4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:51.855672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3406cac-096d-466b-b43b-936cd77f05cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:51.856311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3406cac-096d-466b-b43b-936cd77f05cd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:51.856638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3406cac-096d-466b-b43b-936cd77f05cd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:51.856920 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3406cac-096d-466b-b43b-936cd77f05cd None admin] PATH_INFO: `/v3/groups/1a8a7be2623642c19a79fdd0965fdf22` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:51.858097 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3406cac-096d-466b-b43b-936cd77f05cd None admin] RBAC: Authorizing `identity:get_group(group_id=1a8a7be2623642c19a79fdd0965fdf22)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:51.859582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3406cac-096d-466b-b43b-936cd77f05cd None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:51.861204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 205/412] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:51 2022] GET /identity/v3/groups/1a8a7be2623642c19a79fdd0965fdf22 => generated 229 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:51.862977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b9b35-6e82-412b-bee0-c4d931e2d138 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:51.863348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b9b35-6e82-412b-bee0-c4d931e2d138 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:51.863627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b9b35-6e82-412b-bee0-c4d931e2d138 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:51.864982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 206/413] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:51.867530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b9b35-6e82-412b-bee0-c4d931e2d138 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:51.881132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:51.914671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 208/414] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 1 05:12:51 2022] PUT /identity/v3/projects/d61e52cca9584059bf484cabd52d2808/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8/roles/acfe3eeb91964f62abb1d8a84c36dccc => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:12:51.920306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee752c30-5d13-4fa3-a108-54ccfed5cfdd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:51.920306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee752c30-5d13-4fa3-a108-54ccfed5cfdd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:51.920306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee752c30-5d13-4fa3-a108-54ccfed5cfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:51.939701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ee752c30-5d13-4fa3-a108-54ccfed5cfdd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:51.941764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ee752c30-5d13-4fa3-a108-54ccfed5cfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:52.017959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 209/415] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:52.059499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b9b35-6e82-412b-bee0-c4d931e2d138 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:52.119722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac355fca-3795-4619-a691-818c6064fc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:52.121698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac355fca-3795-4619-a691-818c6064fc89 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:52.121698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac355fca-3795-4619-a691-818c6064fc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:52.121698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac355fca-3795-4619-a691-818c6064fc89 None admin] PATH_INFO: `/v3/projects/61f9dee9572644e580e35da89b6265e9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:52.122063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac355fca-3795-4619-a691-818c6064fc89 None admin] RBAC: Authorizing `identity:get_project(project_id=61f9dee9572644e580e35da89b6265e9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:52.123812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac355fca-3795-4619-a691-818c6064fc89 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:52.125412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 207/416] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:51 2022] GET /identity/v3/projects/61f9dee9572644e580e35da89b6265e9 => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:52.131315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee752c30-5d13-4fa3-a108-54ccfed5cfdd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:52.184223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:52.260702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee752c30-5d13-4fa3-a108-54ccfed5cfdd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:52.301330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb6ff1-07d3-48a7-84fe-97e3983c3456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:52.302021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb6ff1-07d3-48a7-84fe-97e3983c3456 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:52.302295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb6ff1-07d3-48a7-84fe-97e3983c3456 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:52.302561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb6ff1-07d3-48a7-84fe-97e3983c3456 None admin] PATH_INFO: `/v3/projects/61f9dee9572644e580e35da89b6265e9/groups/1a8a7be2623642c19a79fdd0965fdf22/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:52.310107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb6ff1-07d3-48a7-84fe-97e3983c3456 None admin] RBAC: Authorizing `identity:create_grant(project_id=61f9dee9572644e580e35da89b6265e9, group_id=1a8a7be2623642c19a79fdd0965fdf22, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:52.311543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb6ff1-07d3-48a7-84fe-97e3983c3456 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:52.322036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac355fca-3795-4619-a691-818c6064fc89 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:52.392001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 210/417] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 1 05:12:52 2022] PUT /identity/v3/projects/61f9dee9572644e580e35da89b6265e9/groups/1a8a7be2623642c19a79fdd0965fdf22/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:12:52.412878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:52.445922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac355fca-3795-4619-a691-818c6064fc89 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:52.483878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a8e40-b0af-42aa-bb1a-683e7349341c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:52.484519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a8e40-b0af-42aa-bb1a-683e7349341c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:52.484794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a8e40-b0af-42aa-bb1a-683e7349341c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:52.485062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a8e40-b0af-42aa-bb1a-683e7349341c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:52.498320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a8e40-b0af-42aa-bb1a-683e7349341c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-c43a8e40-b0af-42aa-bb1a-683e7349341c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:52.504372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:52.505235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:52.506093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 208/418] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:52.511815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb6ff1-07d3-48a7-84fe-97e3983c3456 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:52.642963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb6ff1-07d3-48a7-84fe-97e3983c3456 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:52.645075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:52.681736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f634049-cb98-43a7-a4f4-9f1c2104511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:52.682355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f634049-cb98-43a7-a4f4-9f1c2104511a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:52.682635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f634049-cb98-43a7-a4f4-9f1c2104511a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:52.682909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f634049-cb98-43a7-a4f4-9f1c2104511a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:52.683661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f634049-cb98-43a7-a4f4-9f1c2104511a None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:52.684018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f634049-cb98-43a7-a4f4-9f1c2104511a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:52.685480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f634049-cb98-43a7-a4f4-9f1c2104511a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:52.696052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 211/419] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:52.701671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a8e40-b0af-42aa-bb1a-683e7349341c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:52.839996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a8e40-b0af-42aa-bb1a-683e7349341c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:52.851049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d122bbb-9d79-4ca8-b373-ca324f5c2811 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:52.851330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d122bbb-9d79-4ca8-b373-ca324f5c2811 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:52.851609 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d122bbb-9d79-4ca8-b373-ca324f5c2811 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:52.853512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 212/420] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:12:52.858457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371f08f-fde1-4780-80d6-7ede9a6943bd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:52.858743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371f08f-fde1-4780-80d6-7ede9a6943bd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:52.859018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371f08f-fde1-4780-80d6-7ede9a6943bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:52.888877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5371f08f-fde1-4780-80d6-7ede9a6943bd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:52.890099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5371f08f-fde1-4780-80d6-7ede9a6943bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:52.897723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f22c-b0f4-4045-ba02-12cac2547b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:52.898277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f22c-b0f4-4045-ba02-12cac2547b2b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:52.898550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f22c-b0f4-4045-ba02-12cac2547b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:52.898819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f22c-b0f4-4045-ba02-12cac2547b2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:52.913023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.007454 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 213/421] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:53.009757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039fe18-2350-43cc-b527-fddc3076723e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.010066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039fe18-2350-43cc-b527-fddc3076723e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.010322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039fe18-2350-43cc-b527-fddc3076723e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.011725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 214/422] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:53.016935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3348921-29dd-45c2-a08b-075cf6bd860e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.017221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3348921-29dd-45c2-a08b-075cf6bd860e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.017534 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3348921-29dd-45c2-a08b-075cf6bd860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.019215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8f22c-b0f4-4045-ba02-12cac2547b2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:53.020893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8f22c-b0f4-4045-ba02-12cac2547b2b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:53.058590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f3348921-29dd-45c2-a08b-075cf6bd860e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:53.059830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f3348921-29dd-45c2-a08b-075cf6bd860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:53.141962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 209/423] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:53.148366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f22c-b0f4-4045-ba02-12cac2547b2b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.221448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 215/424] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:53.233925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.306129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f22c-b0f4-4045-ba02-12cac2547b2b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.347437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293beed9-61a8-40eb-903a-57a2085be141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:53.348572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293beed9-61a8-40eb-903a-57a2085be141 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.349047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293beed9-61a8-40eb-903a-57a2085be141 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.349538 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293beed9-61a8-40eb-903a-57a2085be141 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.350931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293beed9-61a8-40eb-903a-57a2085be141 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:53.352576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293beed9-61a8-40eb-903a-57a2085be141 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:53.354736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 210/425] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:53.363676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3348921-29dd-45c2-a08b-075cf6bd860e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.376875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293beed9-61a8-40eb-903a-57a2085be141 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.416884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.553971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293beed9-61a8-40eb-903a-57a2085be141 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.563386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.595914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec30696-0e84-40bc-972f-5bd3acaf9ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:53.598961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec30696-0e84-40bc-972f-5bd3acaf9ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.599444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec30696-0e84-40bc-972f-5bd3acaf9ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.599928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec30696-0e84-40bc-972f-5bd3acaf9ec9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.613738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec30696-0e84-40bc-972f-5bd3acaf9ec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cec30696-0e84-40bc-972f-5bd3acaf9ec9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:53.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:53.620285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:53.621075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:53.624027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 211/426] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:53.629769 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec30696-0e84-40bc-972f-5bd3acaf9ec9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.653311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3348921-29dd-45c2-a08b-075cf6bd860e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.727039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71df5d7-b0b8-4b27-ac61-8fdc62201d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:53.727633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71df5d7-b0b8-4b27-ac61-8fdc62201d9b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.727917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71df5d7-b0b8-4b27-ac61-8fdc62201d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.728195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71df5d7-b0b8-4b27-ac61-8fdc62201d9b None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.732845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71df5d7-b0b8-4b27-ac61-8fdc62201d9b None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:53.734340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71df5d7-b0b8-4b27-ac61-8fdc62201d9b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:53.742587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 216/427] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:53.743484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71df5d7-b0b8-4b27-ac61-8fdc62201d9b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.764880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec30696-0e84-40bc-972f-5bd3acaf9ec9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.809835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af023f-1d56-485b-97bf-1da6eba35061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:53.811234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af023f-1d56-485b-97bf-1da6eba35061 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.811797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af023f-1d56-485b-97bf-1da6eba35061 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.812309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af023f-1d56-485b-97bf-1da6eba35061 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.827375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af023f-1d56-485b-97bf-1da6eba35061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-07af023f-1d56-485b-97bf-1da6eba35061 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:53.834125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:53.835072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:12:53.835926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:53.839352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 212/428] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:53.840469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af023f-1d56-485b-97bf-1da6eba35061 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.884006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.934690 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.946487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71df5d7-b0b8-4b27-ac61-8fdc62201d9b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.980834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af023f-1d56-485b-97bf-1da6eba35061 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:54.019551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2501e88-e8db-44b9-89fb-ce168bb9886a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:54.020228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2501e88-e8db-44b9-89fb-ce168bb9886a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.020444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2501e88-e8db-44b9-89fb-ce168bb9886a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.020755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2501e88-e8db-44b9-89fb-ce168bb9886a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.021384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2501e88-e8db-44b9-89fb-ce168bb9886a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:54.021730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2501e88-e8db-44b9-89fb-ce168bb9886a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:54.023094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2501e88-e8db-44b9-89fb-ce168bb9886a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:54.036746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 217/429] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:54.036746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2501e88-e8db-44b9-89fb-ce168bb9886a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:54.039056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0530f9-567d-4aba-accf-0de9496338d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:54.039950 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0530f9-567d-4aba-accf-0de9496338d7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.040455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0530f9-567d-4aba-accf-0de9496338d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.040941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0530f9-567d-4aba-accf-0de9496338d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.041921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0530f9-567d-4aba-accf-0de9496338d7 None admin] RBAC: Adding query filter params (group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, scope.project.id=812094fb94404914876df4528f073add) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:54.042449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0530f9-567d-4aba-accf-0de9496338d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:54.044740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0530f9-567d-4aba-accf-0de9496338d7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:54.054980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 213/430] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/role_assignments?group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8&scope.project.id=812094fb94404914876df4528f073add&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 611 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:54.056490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0530f9-567d-4aba-accf-0de9496338d7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:54.176278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:54.189001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0530f9-567d-4aba-accf-0de9496338d7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:54.236814 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a20455-472e-4e42-a927-730dbfea5dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:54.239107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a20455-472e-4e42-a927-730dbfea5dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.239731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a20455-472e-4e42-a927-730dbfea5dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.241579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a20455-472e-4e42-a927-730dbfea5dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.250707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2501e88-e8db-44b9-89fb-ce168bb9886a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:54.313674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013796f-36c7-4790-8460-7fd59eec9671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:54.314720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013796f-36c7-4790-8460-7fd59eec9671 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.315513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013796f-36c7-4790-8460-7fd59eec9671 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.315873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013796f-36c7-4790-8460-7fd59eec9671 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.316832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013796f-36c7-4790-8460-7fd59eec9671 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:54.317121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013796f-36c7-4790-8460-7fd59eec9671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:54.318769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013796f-36c7-4790-8460-7fd59eec9671 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:54.328277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a20455-472e-4e42-a927-730dbfea5dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:54.329582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 218/431] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:12:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:54.330369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a20455-472e-4e42-a927-730dbfea5dc3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:54.335082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013796f-36c7-4790-8460-7fd59eec9671 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:54.361625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:54.454999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:54.471884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 214/432] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:54.473152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828b9c-4384-4c24-8284-9c57a8f42ea8 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.475177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828b9c-4384-4c24-8284-9c57a8f42ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.475177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828b9c-4384-4c24-8284-9c57a8f42ea8 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.475177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 215/433] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:54.480740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db828b9c-4384-4c24-8284-9c57a8f42ea8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:54.485766 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013796f-36c7-4790-8460-7fd59eec9671 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:54.533337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803a704-038f-4729-8c10-48805e7fb09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:54.534519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803a704-038f-4729-8c10-48805e7fb09c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.534519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803a704-038f-4729-8c10-48805e7fb09c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.534708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803a704-038f-4729-8c10-48805e7fb09c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.611506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db828b9c-4384-4c24-8284-9c57a8f42ea8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:54.616536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803a704-038f-4729-8c10-48805e7fb09c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:54.618050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803a704-038f-4729-8c10-48805e7fb09c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:54.639844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:54.651629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c85f1-c679-484c-8111-e321783280d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:54.654916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c85f1-c679-484c-8111-e321783280d3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.654916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c85f1-c679-484c-8111-e321783280d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.654916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c85f1-c679-484c-8111-e321783280d3 None admin] PATH_INFO: `/v3/projects/812094fb94404914876df4528f073add` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.654916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c85f1-c679-484c-8111-e321783280d3 None admin] RBAC: Authorizing `identity:get_project(project_id=812094fb94404914876df4528f073add)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:54.655692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c85f1-c679-484c-8111-e321783280d3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:54.657375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 216/434] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:54 2022] GET /identity/v3/projects/812094fb94404914876df4528f073add => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:54.664089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7479e-ff4f-4798-b874-6a4e629ebac3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.664375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7479e-ff4f-4798-b874-6a4e629ebac3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.664648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7479e-ff4f-4798-b874-6a4e629ebac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.686292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-00d7479e-ff4f-4798-b874-6a4e629ebac3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:54.688071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-00d7479e-ff4f-4798-b874-6a4e629ebac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:54.743506 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 219/435] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:54.744755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803a704-038f-4729-8c10-48805e7fb09c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:54.763780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:54.777698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 217/436] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:54.783240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7479e-ff4f-4798-b874-6a4e629ebac3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:54.853121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803a704-038f-4729-8c10-48805e7fb09c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:54.871179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:54.885420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7479e-ff4f-4798-b874-6a4e629ebac3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:54.890039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d938b08-5fdd-4671-b2ca-32c392c5ff5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:54.890676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d938b08-5fdd-4671-b2ca-32c392c5ff5f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.890958 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d938b08-5fdd-4671-b2ca-32c392c5ff5f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.891229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d938b08-5fdd-4671-b2ca-32c392c5ff5f None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.892379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d938b08-5fdd-4671-b2ca-32c392c5ff5f None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:54.894691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d938b08-5fdd-4671-b2ca-32c392c5ff5f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:54.901508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 220/437] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:54 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:54.901508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e84e7b-bd85-4074-82cc-7346f6d3579d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.901508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e84e7b-bd85-4074-82cc-7346f6d3579d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.901508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e84e7b-bd85-4074-82cc-7346f6d3579d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.901508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 221/438] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:54.902455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e84e7b-bd85-4074-82cc-7346f6d3579d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:54.928242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52506a-a096-46b5-9b3d-2c44849b3e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:54.928867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52506a-a096-46b5-9b3d-2c44849b3e79 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.929143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52506a-a096-46b5-9b3d-2c44849b3e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.929416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52506a-a096-46b5-9b3d-2c44849b3e79 None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.930573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52506a-a096-46b5-9b3d-2c44849b3e79 None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:54.931986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52506a-a096-46b5-9b3d-2c44849b3e79 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:54.933613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 218/439] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:54 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:54.934755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753221c4-1c49-41f9-a6c6-02b795305c58 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:54.935038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753221c4-1c49-41f9-a6c6-02b795305c58 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:54.935307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753221c4-1c49-41f9-a6c6-02b795305c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:54.953133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-753221c4-1c49-41f9-a6c6-02b795305c58 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:54.954324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-753221c4-1c49-41f9-a6c6-02b795305c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:55.007745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:55.020403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e84e7b-bd85-4074-82cc-7346f6d3579d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:55.031140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 219/440] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:55.031140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753221c4-1c49-41f9-a6c6-02b795305c58 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:55.058301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dbf20-f2d9-4bbb-a90e-e46f53e28c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:55.058972 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dbf20-f2d9-4bbb-a90e-e46f53e28c51 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.059398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dbf20-f2d9-4bbb-a90e-e46f53e28c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.059718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dbf20-f2d9-4bbb-a90e-e46f53e28c51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.060505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094dbf20-f2d9-4bbb-a90e-e46f53e28c51 None admin] RBAC: Adding query filter params (group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role.id=acfe3eeb91964f62abb1d8a84c36dccc, scope.project.id=812094fb94404914876df4528f073add) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:55.060815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094dbf20-f2d9-4bbb-a90e-e46f53e28c51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:55.062289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094dbf20-f2d9-4bbb-a90e-e46f53e28c51 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:55.068985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 222/441] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 1 05:12:54 2022] GET /identity/v3/role_assignments?group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8&scope.project.id=812094fb94404914876df4528f073add&role.id=acfe3eeb91964f62abb1d8a84c36dccc => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:55.069892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dbf20-f2d9-4bbb-a90e-e46f53e28c51 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:55.128874 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:55.180782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753221c4-1c49-41f9-a6c6-02b795305c58 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:55.197363 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dbf20-f2d9-4bbb-a90e-e46f53e28c51 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:55.224904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2caaa34-a816-4420-bfbf-495cec5b5cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:55.227598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caaa34-a816-4420-bfbf-495cec5b5cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.227598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caaa34-a816-4420-bfbf-495cec5b5cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.227598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caaa34-a816-4420-bfbf-495cec5b5cb4 None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.227598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2caaa34-a816-4420-bfbf-495cec5b5cb4 None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:55.228674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2caaa34-a816-4420-bfbf-495cec5b5cb4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:55.230597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 220/442] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:55 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:55.236625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2caaa34-a816-4420-bfbf-495cec5b5cb4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:55.239803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8248cd-8025-48bf-ade2-c1ea2300471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:55.240623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8248cd-8025-48bf-ade2-c1ea2300471c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.241836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8248cd-8025-48bf-ade2-c1ea2300471c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.242121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8248cd-8025-48bf-ade2-c1ea2300471c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.256515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8248cd-8025-48bf-ade2-c1ea2300471c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-2a8248cd-8025-48bf-ade2-c1ea2300471c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:55.262135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:55.263041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:12:55.263833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 223/443] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:12:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:55.271355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8248cd-8025-48bf-ade2-c1ea2300471c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:55.306097 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:55.359066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2caaa34-a816-4420-bfbf-495cec5b5cb4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:55.397695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8248cd-8025-48bf-ade2-c1ea2300471c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:55.401377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c01e6-1e8a-45ff-8cf9-0d763dafb2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:55.402032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c01e6-1e8a-45ff-8cf9-0d763dafb2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.402313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c01e6-1e8a-45ff-8cf9-0d763dafb2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.402585 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c01e6-1e8a-45ff-8cf9-0d763dafb2c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.403313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c01e6-1e8a-45ff-8cf9-0d763dafb2c4 None admin] RBAC: Adding query filter params (group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75, scope.project.id=d61e52cca9584059bf484cabd52d2808) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:55.403622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c01e6-1e8a-45ff-8cf9-0d763dafb2c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:55.405060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c01e6-1e8a-45ff-8cf9-0d763dafb2c4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:55.408257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:55.417482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 221/444] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 1 05:12:55 2022] GET /identity/v3/role_assignments?group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:55.417482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c01e6-1e8a-45ff-8cf9-0d763dafb2c4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:55.458103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6a2e0-701c-43ef-bb0c-48f77a4eb307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:55.458743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6a2e0-701c-43ef-bb0c-48f77a4eb307 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.459027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6a2e0-701c-43ef-bb0c-48f77a4eb307 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.459303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6a2e0-701c-43ef-bb0c-48f77a4eb307 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.460327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6a2e0-701c-43ef-bb0c-48f77a4eb307 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:55.461618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6a2e0-701c-43ef-bb0c-48f77a4eb307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:55.463143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6a2e0-701c-43ef-bb0c-48f77a4eb307 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:55.473894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 224/445] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:55.481289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6a2e0-701c-43ef-bb0c-48f77a4eb307 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:55.552252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c01e6-1e8a-45ff-8cf9-0d763dafb2c4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:55.572652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:55.596596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb7deb-55a5-4d1f-86ed-ac9dbaa11e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:55.597238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb7deb-55a5-4d1f-86ed-ac9dbaa11e0c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.597855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb7deb-55a5-4d1f-86ed-ac9dbaa11e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.598041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb7deb-55a5-4d1f-86ed-ac9dbaa11e0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.610470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb7deb-55a5-4d1f-86ed-ac9dbaa11e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:55.615188 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6a2e0-701c-43ef-bb0c-48f77a4eb307 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e8bb7deb-55a5-4d1f-86ed-ac9dbaa11e0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:55.616649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:55.617614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:55.618443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 222/446] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:55.626457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb7deb-55a5-4d1f-86ed-ac9dbaa11e0c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:55.666389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c574e-8247-43df-aeca-914e4af7ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:55.666940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c574e-8247-43df-aeca-914e4af7ec32 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.667218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c574e-8247-43df-aeca-914e4af7ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.667488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c574e-8247-43df-aeca-914e4af7ec32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.722398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:55.757748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb7deb-55a5-4d1f-86ed-ac9dbaa11e0c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:55.759396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c574e-8247-43df-aeca-914e4af7ec32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:55.760896 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c574e-8247-43df-aeca-914e4af7ec32 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:55.798809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fdfb2-e556-45b3-865b-d3936fc60ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:55.799528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fdfb2-e556-45b3-865b-d3936fc60ddd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.799966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fdfb2-e556-45b3-865b-d3936fc60ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.800254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fdfb2-e556-45b3-865b-d3936fc60ddd None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.800921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8fdfb2-e556-45b3-865b-d3936fc60ddd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:55.801226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8fdfb2-e556-45b3-865b-d3936fc60ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:55.802633 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8fdfb2-e556-45b3-865b-d3936fc60ddd None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:55.810825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 223/447] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:55.816877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fdfb2-e556-45b3-865b-d3936fc60ddd None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:55.858554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:55.869040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 225/448] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:12:55.874883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c574e-8247-43df-aeca-914e4af7ec32 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:55.919963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fdfb2-e556-45b3-865b-d3936fc60ddd None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:55.944252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:55.953906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a600c8-6528-4a7a-9573-ee57a436848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:55.954575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a600c8-6528-4a7a-9573-ee57a436848b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.954854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a600c8-6528-4a7a-9573-ee57a436848b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.955238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a600c8-6528-4a7a-9573-ee57a436848b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.975867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c574e-8247-43df-aeca-914e4af7ec32 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.009908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01eecd9-18f6-414b-8e41-60d3f799e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.010532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01eecd9-18f6-414b-8e41-60d3f799e8af None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.010813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01eecd9-18f6-414b-8e41-60d3f799e8af None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.011090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01eecd9-18f6-414b-8e41-60d3f799e8af None admin] PATH_INFO: `/v3/projects/d61e52cca9584059bf484cabd52d2808` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.012215 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01eecd9-18f6-414b-8e41-60d3f799e8af None admin] RBAC: Authorizing `identity:get_project(project_id=d61e52cca9584059bf484cabd52d2808)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.013673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01eecd9-18f6-414b-8e41-60d3f799e8af None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.015329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 226/449] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:55 2022] GET /identity/v3/projects/d61e52cca9584059bf484cabd52d2808 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:56.021347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01eecd9-18f6-414b-8e41-60d3f799e8af None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:56.023831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a600c8-6528-4a7a-9573-ee57a436848b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.025285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a600c8-6528-4a7a-9573-ee57a436848b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.064305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:56.120262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01eecd9-18f6-414b-8e41-60d3f799e8af None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.124484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 224/450] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:12:56.132217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a600c8-6528-4a7a-9573-ee57a436848b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:56.154812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad93f1-467f-4b0f-a3ae-ab30fe4def8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.155445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad93f1-467f-4b0f-a3ae-ab30fe4def8b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.155722 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad93f1-467f-4b0f-a3ae-ab30fe4def8b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.155994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad93f1-467f-4b0f-a3ae-ab30fe4def8b None admin] PATH_INFO: `/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.157118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad93f1-467f-4b0f-a3ae-ab30fe4def8b None admin] RBAC: Authorizing `identity:get_group(group_id=97bf8e8e13f34e7e9c79e4ba51e8c6f8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.158733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad93f1-467f-4b0f-a3ae-ab30fe4def8b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.160518 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 227/451] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:56 2022] GET /identity/v3/groups/97bf8e8e13f34e7e9c79e4ba51e8c6f8 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:56.165994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad93f1-467f-4b0f-a3ae-ab30fe4def8b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:56.174583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:56.232922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a600c8-6528-4a7a-9573-ee57a436848b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.266488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6add9-9dc3-48a1-af0a-92c1bd86f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.267112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6add9-9dc3-48a1-af0a-92c1bd86f991 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.267390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6add9-9dc3-48a1-af0a-92c1bd86f991 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.267662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6add9-9dc3-48a1-af0a-92c1bd86f991 None admin] PATH_INFO: `/v3/projects/61f9dee9572644e580e35da89b6265e9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.268588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad93f1-467f-4b0f-a3ae-ab30fe4def8b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.269303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6add9-9dc3-48a1-af0a-92c1bd86f991 None admin] RBAC: Authorizing `identity:get_project(project_id=61f9dee9572644e580e35da89b6265e9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.270791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6add9-9dc3-48a1-af0a-92c1bd86f991 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.272421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 225/452] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:56 2022] GET /identity/v3/projects/61f9dee9572644e580e35da89b6265e9 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:56.277820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6add9-9dc3-48a1-af0a-92c1bd86f991 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:56.288622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:56.302558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac555c9-5a6a-4b76-95b4-a969cfa62735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.303158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac555c9-5a6a-4b76-95b4-a969cfa62735 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.303433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac555c9-5a6a-4b76-95b4-a969cfa62735 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.303703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac555c9-5a6a-4b76-95b4-a969cfa62735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.304422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac555c9-5a6a-4b76-95b4-a969cfa62735 None admin] RBAC: Adding query filter params (group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8, role.id=acfe3eeb91964f62abb1d8a84c36dccc, scope.project.id=d61e52cca9584059bf484cabd52d2808) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:56.304729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac555c9-5a6a-4b76-95b4-a969cfa62735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.306204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac555c9-5a6a-4b76-95b4-a969cfa62735 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.311930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 228/453] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 1 05:12:56 2022] GET /identity/v3/role_assignments?group.id=97bf8e8e13f34e7e9c79e4ba51e8c6f8&scope.project.id=d61e52cca9584059bf484cabd52d2808&role.id=acfe3eeb91964f62abb1d8a84c36dccc => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:56.382438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6add9-9dc3-48a1-af0a-92c1bd86f991 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.424770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a9fc7-5e18-43b9-8bf1-b475702405ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.425340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a9fc7-5e18-43b9-8bf1-b475702405ea None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.425649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a9fc7-5e18-43b9-8bf1-b475702405ea None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.425917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a9fc7-5e18-43b9-8bf1-b475702405ea None admin] PATH_INFO: `/v3/groups/1a8a7be2623642c19a79fdd0965fdf22` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.427084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a9fc7-5e18-43b9-8bf1-b475702405ea None admin] RBAC: Authorizing `identity:get_group(group_id=1a8a7be2623642c19a79fdd0965fdf22)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.428532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a9fc7-5e18-43b9-8bf1-b475702405ea None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.430136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 226/454] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 1 05:12:56 2022] GET /identity/v3/groups/1a8a7be2623642c19a79fdd0965fdf22 => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:56.436188 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac555c9-5a6a-4b76-95b4-a969cfa62735 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:56.508282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:56.540660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac555c9-5a6a-4b76-95b4-a969cfa62735 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.574437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04332ae5-b141-4012-8233-6917cc7f9fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.575311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04332ae5-b141-4012-8233-6917cc7f9fdd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.575747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04332ae5-b141-4012-8233-6917cc7f9fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.576172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04332ae5-b141-4012-8233-6917cc7f9fdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.576900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04332ae5-b141-4012-8233-6917cc7f9fdd None admin] RBAC: Adding query filter params (group.id=1a8a7be2623642c19a79fdd0965fdf22, role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=61f9dee9572644e580e35da89b6265e9) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:56.577211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04332ae5-b141-4012-8233-6917cc7f9fdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.578729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04332ae5-b141-4012-8233-6917cc7f9fdd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.584778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 229/455] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 1 05:12:56 2022] GET /identity/v3/role_assignments?group.id=1a8a7be2623642c19a79fdd0965fdf22&scope.project.id=61f9dee9572644e580e35da89b6265e9&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:58.950677 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261a43e-aabb-411f-a31b-9c4a45c09e71 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:58.951603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261a43e-aabb-411f-a31b-9c4a45c09e71 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:58.952047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261a43e-aabb-411f-a31b-9c4a45c09e71 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:58.953876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 227/456] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:58.959358 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29cf02-b2e9-423b-991c-db970ef2d7ad None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:58.960158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29cf02-b2e9-423b-991c-db970ef2d7ad None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:58.960626 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29cf02-b2e9-423b-991c-db970ef2d7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:58.986556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-eb29cf02-b2e9-423b-991c-db970ef2d7ad None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:58.988019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eb29cf02-b2e9-423b-991c-db970ef2d7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:59.075159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 230/457] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:59.356165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4752b4-7dca-43c3-b686-5511164873af None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.356837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4752b4-7dca-43c3-b686-5511164873af None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.357300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4752b4-7dca-43c3-b686-5511164873af None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.359168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 228/458] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:59.363761 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6811c5-9e1f-4e31-a4d7-5874298b4e82 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.363913 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37de10e-640d-4364-9bad-f4b242732c83 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.367104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37de10e-640d-4364-9bad-f4b242732c83 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.367250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6811c5-9e1f-4e31-a4d7-5874298b4e82 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.367340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37de10e-640d-4364-9bad-f4b242732c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.367413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6811c5-9e1f-4e31-a4d7-5874298b4e82 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.367413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 231/459] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:59.370882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc48f-d888-40df-b428-008e17ec2ce3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.371164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc48f-d888-40df-b428-008e17ec2ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.371435 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc48f-d888-40df-b428-008e17ec2ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.393492 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a37de10e-640d-4364-9bad-f4b242732c83 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:59.393492 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a37de10e-640d-4364-9bad-f4b242732c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:59.399110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-06edc48f-d888-40df-b428-008e17ec2ce3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:59.400270 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-06edc48f-d888-40df-b428-008e17ec2ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:59.505707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:59.511710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 229/460] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:59.516406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37de10e-640d-4364-9bad-f4b242732c83 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:59.518105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 232/461] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:12:59.535989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:59.582623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3274ae5-5f6a-4fa0-9a76-e7873484e817 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.582965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3274ae5-5f6a-4fa0-9a76-e7873484e817 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.583168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3274ae5-5f6a-4fa0-9a76-e7873484e817 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.584739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 233/462] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:59.590154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdeac0-fcc3-481c-ae02-b5754db91fad None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.590437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdeac0-fcc3-481c-ae02-b5754db91fad None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.590894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdeac0-fcc3-481c-ae02-b5754db91fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.628932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f1bdeac0-fcc3-481c-ae02-b5754db91fad None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:59.630063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f1bdeac0-fcc3-481c-ae02-b5754db91fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:59.697365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37de10e-640d-4364-9bad-f4b242732c83 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:59.764879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 234/463] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:12:59.767273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd225d6c-7963-4758-a9f6-6592232435b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:59.768118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd225d6c-7963-4758-a9f6-6592232435b8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.768600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd225d6c-7963-4758-a9f6-6592232435b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.768926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd225d6c-7963-4758-a9f6-6592232435b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.780485 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd225d6c-7963-4758-a9f6-6592232435b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:59.781975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd225d6c-7963-4758-a9f6-6592232435b8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:59.802139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdeac0-fcc3-481c-ae02-b5754db91fad None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cd225d6c-7963-4758-a9f6-6592232435b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:12:59.815799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:59.816729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:12:59.817938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:12:59.817938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:59.817938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:12:59.817938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 230/464] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:59.824220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd225d6c-7963-4758-a9f6-6592232435b8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:59.895648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:59.932826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdeac0-fcc3-481c-ae02-b5754db91fad None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:59.943054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd225d6c-7963-4758-a9f6-6592232435b8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:59.958845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:59.970047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d6714-f876-4e1a-b0d7-9376f3e0566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:59.973166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d6714-f876-4e1a-b0d7-9376f3e0566f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.973166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d6714-f876-4e1a-b0d7-9376f3e0566f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.973166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d6714-f876-4e1a-b0d7-9376f3e0566f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.979781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0e6e3-dcde-43eb-a3df-da42996a5629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:59.980877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e6e3-dcde-43eb-a3df-da42996a5629 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.981300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e6e3-dcde-43eb-a3df-da42996a5629 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.981746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e6e3-dcde-43eb-a3df-da42996a5629 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.982549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0e6e3-dcde-43eb-a3df-da42996a5629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:59.982993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0e6e3-dcde-43eb-a3df-da42996a5629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:59.983197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d6714-f876-4e1a-b0d7-9376f3e0566f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:59.984657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d6714-f876-4e1a-b0d7-9376f3e0566f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:59.984840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0e6e3-dcde-43eb-a3df-da42996a5629 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-616d6714-f876-4e1a-b0d7-9376f3e0566f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:00.004667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:00.005833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:00.006693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:00.006693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:00.006693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:00.006693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 235/465] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:00.006983 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d6714-f876-4e1a-b0d7-9376f3e0566f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:00.011393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 231/466] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:00.012486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0e6e3-dcde-43eb-a3df-da42996a5629 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:00.120192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0e6e3-dcde-43eb-a3df-da42996a5629 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:00.120729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d6714-f876-4e1a-b0d7-9376f3e0566f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:00.138686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:00.156149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf6f21-5771-4988-a659-ed1c30fac648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:00.156587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1635eb-33d1-4829-9679-093f0e4b2c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:00.156892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf6f21-5771-4988-a659-ed1c30fac648 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.157087 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf6f21-5771-4988-a659-ed1c30fac648 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.157180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1635eb-33d1-4829-9679-093f0e4b2c48 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.157360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf6f21-5771-4988-a659-ed1c30fac648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.157593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1635eb-33d1-4829-9679-093f0e4b2c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.157762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1635eb-33d1-4829-9679-093f0e4b2c48 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.159001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1635eb-33d1-4829-9679-093f0e4b2c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:00.159306 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1635eb-33d1-4829-9679-093f0e4b2c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:00.160892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1635eb-33d1-4829-9679-093f0e4b2c48 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.170863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf6f21-5771-4988-a659-ed1c30fac648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:00.172482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf6f21-5771-4988-a659-ed1c30fac648 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.177751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 232/467] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:00.177751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1635eb-33d1-4829-9679-093f0e4b2c48 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-82cf6f21-5771-4988-a659-ed1c30fac648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:00.192190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:00.194772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:00.195909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:00.195909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:00.195909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:00.195909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 236/468] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:00.196982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf6f21-5771-4988-a659-ed1c30fac648 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:00.226720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:00.286655 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1635eb-33d1-4829-9679-093f0e4b2c48 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:00.302801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf6f21-5771-4988-a659-ed1c30fac648 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:00.326381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee382-79e0-44eb-841d-a870b87f8a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:00.326996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee382-79e0-44eb-841d-a870b87f8a0f None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.327275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee382-79e0-44eb-841d-a870b87f8a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.327549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee382-79e0-44eb-841d-a870b87f8a0f None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.328191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bee382-79e0-44eb-841d-a870b87f8a0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:00.329657 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bee382-79e0-44eb-841d-a870b87f8a0f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.339654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e9584-a60e-41ff-9085-f575c2968980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:00.340121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e9584-a60e-41ff-9085-f575c2968980 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.340396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e9584-a60e-41ff-9085-f575c2968980 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.341001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e9584-a60e-41ff-9085-f575c2968980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.352096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e9584-a60e-41ff-9085-f575c2968980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:00.353556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e9584-a60e-41ff-9085-f575c2968980 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-164e9584-a60e-41ff-9085-f575c2968980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:00.367880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:00.369039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:00.369981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:00.369981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:00.369981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:00.369981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 237/469] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:00.371898 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e9584-a60e-41ff-9085-f575c2968980 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:00.399781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 233/470] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:13:00 2022] POST /identity/v3/users => generated 246 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:00.400604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee382-79e0-44eb-841d-a870b87f8a0f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:00.450719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:00.490055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:00.513970 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e9584-a60e-41ff-9085-f575c2968980 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:00.529427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee382-79e0-44eb-841d-a870b87f8a0f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:00.558368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70811ae-55c5-4b61-8fcf-505c9b12f14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:00.558966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70811ae-55c5-4b61-8fcf-505c9b12f14d None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.559420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70811ae-55c5-4b61-8fcf-505c9b12f14d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.559545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70811ae-55c5-4b61-8fcf-505c9b12f14d None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.560148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70811ae-55c5-4b61-8fcf-505c9b12f14d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:00.561618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70811ae-55c5-4b61-8fcf-505c9b12f14d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.569272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19bfef-dc9d-4151-854e-474d9f4af97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:00.570349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19bfef-dc9d-4151-854e-474d9f4af97c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.570631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19bfef-dc9d-4151-854e-474d9f4af97c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.570902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19bfef-dc9d-4151-854e-474d9f4af97c None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.572012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19bfef-dc9d-4151-854e-474d9f4af97c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:00.572310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19bfef-dc9d-4151-854e-474d9f4af97c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:00.574859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19bfef-dc9d-4151-854e-474d9f4af97c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.586468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 234/471] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:00.587356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19bfef-dc9d-4151-854e-474d9f4af97c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:00.644463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 238/472] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:13:00 2022] POST /identity/v3/users => generated 248 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:00.645794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd50556-3c62-4d26-9489-7edfc15556cf None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.646079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd50556-3c62-4d26-9489-7edfc15556cf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.646352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd50556-3c62-4d26-9489-7edfc15556cf None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.651611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 239/473] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:00.651611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd50556-3c62-4d26-9489-7edfc15556cf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:00.696763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:00.705839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19bfef-dc9d-4151-854e-474d9f4af97c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:00.766215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2120a0e-0862-4c5c-9031-44853c42385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:00.768537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2120a0e-0862-4c5c-9031-44853c42385f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.769583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2120a0e-0862-4c5c-9031-44853c42385f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.770043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2120a0e-0862-4c5c-9031-44853c42385f None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.771321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2120a0e-0862-4c5c-9031-44853c42385f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:00.772337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2120a0e-0862-4c5c-9031-44853c42385f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:00.774456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2120a0e-0862-4c5c-9031-44853c42385f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.783030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd50556-3c62-4d26-9489-7edfc15556cf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:00.791589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 235/474] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:00.793326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327f57-888d-4b42-a0c7-bcddd4410388 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.794168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327f57-888d-4b42-a0c7-bcddd4410388 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.794599 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327f57-888d-4b42-a0c7-bcddd4410388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.816852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-74327f57-888d-4b42-a0c7-bcddd4410388 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:00.818399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-74327f57-888d-4b42-a0c7-bcddd4410388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:00.832041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc5527-f333-463b-a776-fa85a3b23e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:00.832744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc5527-f333-463b-a776-fa85a3b23e94 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.833029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc5527-f333-463b-a776-fa85a3b23e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.833301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc5527-f333-463b-a776-fa85a3b23e94 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.834005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc5527-f333-463b-a776-fa85a3b23e94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:00.835468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc5527-f333-463b-a776-fa85a3b23e94 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.844527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:00.899392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 236/475] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:00.900248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74327f57-888d-4b42-a0c7-bcddd4410388 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:00.943517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 240/476] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:13:00 2022] POST /identity/v3/users => generated 248 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:01.000780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:01.023388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74327f57-888d-4b42-a0c7-bcddd4410388 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:01.063276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbdc1e-84f8-47f5-a329-6b1f11fc8226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:01.063882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbdc1e-84f8-47f5-a329-6b1f11fc8226 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:01.064154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbdc1e-84f8-47f5-a329-6b1f11fc8226 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:01.065023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbdc1e-84f8-47f5-a329-6b1f11fc8226 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:01.065977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbdc1e-84f8-47f5-a329-6b1f11fc8226 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:01.067611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbdc1e-84f8-47f5-a329-6b1f11fc8226 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:01.198811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 237/477] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:13:00 2022] POST /identity/v3/users => generated 249 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:01.260857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc5527-f333-463b-a776-fa85a3b23e94 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:01.374762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc5527-f333-463b-a776-fa85a3b23e94 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:01.411251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c413bfa-e71c-484b-b84d-e6d210a7db28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:01.411867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c413bfa-e71c-484b-b84d-e6d210a7db28 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:01.412144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c413bfa-e71c-484b-b84d-e6d210a7db28 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:01.412412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c413bfa-e71c-484b-b84d-e6d210a7db28 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:01.413124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c413bfa-e71c-484b-b84d-e6d210a7db28 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:01.414621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c413bfa-e71c-484b-b84d-e6d210a7db28 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:01.443474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.sql.core [None req-7c413bfa-e71c-484b-b84d-e6d210a7db28 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 05:13:01.443474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 05:13:01.443474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [parameters: {'id': '5af4f66301df4ea7b00608151c1b38b7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 05:13:01.443474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81574) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7c413bfa-e71c-484b-b84d-e6d210a7db28 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application result.read() Sep 01 05:13:01.462544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:13:01.463482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application yield resource Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise self.value Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application session.commit() Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 05:13:01.464419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self.session.flush() Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application compat.raise_( Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 05:13:01.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:13:01.467923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application result.read() Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:13:01.469917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [parameters: {'id': '5af4f66301df4ea7b00608151c1b38b7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:01.472638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 05:13:01.474753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:01.476871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 241/478] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 1 05:13:01 2022] POST /identity/v3/roles => generated 144 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:01.477017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbdc1e-84f8-47f5-a329-6b1f11fc8226 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:01.599908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbdc1e-84f8-47f5-a329-6b1f11fc8226 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:01.641156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5902f-9c95-407c-9252-48be0eb44769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:01.641821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5902f-9c95-407c-9252-48be0eb44769 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:01.642097 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5902f-9c95-407c-9252-48be0eb44769 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:01.642366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5902f-9c95-407c-9252-48be0eb44769 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:01.659686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:01.663770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5902f-9c95-407c-9252-48be0eb44769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-bda5902f-9c95-407c-9252-48be0eb44769 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:01.668592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:01.670668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:13:01.672604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:01.674782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 238/479] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:13:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:01.677576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c413bfa-e71c-484b-b84d-e6d210a7db28 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:01.763500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:01.815891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c413bfa-e71c-484b-b84d-e6d210a7db28 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:01.858707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189ad02-6cde-4234-8374-ec4ef899ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:01.859246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189ad02-6cde-4234-8374-ec4ef899ffa9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:01.859519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189ad02-6cde-4234-8374-ec4ef899ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:01.859785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189ad02-6cde-4234-8374-ec4ef899ffa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:01.860445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189ad02-6cde-4234-8374-ec4ef899ffa9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:01.860743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189ad02-6cde-4234-8374-ec4ef899ffa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:01.862263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189ad02-6cde-4234-8374-ec4ef899ffa9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:01.870908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 242/480] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:02.592163 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df77f9-3edd-4d90-81f6-6d4a03ef6626 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:02.592706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df77f9-3edd-4d90-81f6-6d4a03ef6626 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:02.592706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df77f9-3edd-4d90-81f6-6d4a03ef6626 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:02.594303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 239/481] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:02.600175 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058fe6b-25d6-4835-86de-e2d03574435c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:02.600587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058fe6b-25d6-4835-86de-e2d03574435c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:02.600766 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058fe6b-25d6-4835-86de-e2d03574435c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:02.625907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1058fe6b-25d6-4835-86de-e2d03574435c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:02.627698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1058fe6b-25d6-4835-86de-e2d03574435c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:02.717772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 243/482] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:02.852709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0e312-c88c-482a-a4a4-f37c992ebf59 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:02.852881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0e312-c88c-482a-a4a4-f37c992ebf59 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:02.853152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0e312-c88c-482a-a4a4-f37c992ebf59 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:02.854515 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 240/483] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:02.859893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f779930-8abe-428f-9943-47953a925caf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:02.860181 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f779930-8abe-428f-9943-47953a925caf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:02.860453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f779930-8abe-428f-9943-47953a925caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:02.884424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1f779930-8abe-428f-9943-47953a925caf None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:02.885963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1f779930-8abe-428f-9943-47953a925caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:02.931890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:02.956655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 244/484] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:03.071702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0e312-c88c-482a-a4a4-f37c992ebf59 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:03.203191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0e312-c88c-482a-a4a4-f37c992ebf59 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:03.247075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472eba83-9c1a-40b2-87da-7cdd7f3c899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:03.247715 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472eba83-9c1a-40b2-87da-7cdd7f3c899d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:03.247995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472eba83-9c1a-40b2-87da-7cdd7f3c899d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:03.248269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472eba83-9c1a-40b2-87da-7cdd7f3c899d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:03.264630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472eba83-9c1a-40b2-87da-7cdd7f3c899d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-472eba83-9c1a-40b2-87da-7cdd7f3c899d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:03.269953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:03.271034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:03.272032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 241/485] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:03.278821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f779930-8abe-428f-9943-47953a925caf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:03.449432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f779930-8abe-428f-9943-47953a925caf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:03.454081 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472eba83-9c1a-40b2-87da-7cdd7f3c899d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:03.508105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabb050-3c50-4004-bf67-2389f2c72a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:03.508968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabb050-3c50-4004-bf67-2389f2c72a4a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:03.511089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabb050-3c50-4004-bf67-2389f2c72a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:03.511509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabb050-3c50-4004-bf67-2389f2c72a4a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:03.512322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabb050-3c50-4004-bf67-2389f2c72a4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:03.514112 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabb050-3c50-4004-bf67-2389f2c72a4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:03.515642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabb050-3c50-4004-bf67-2389f2c72a4a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:03.540568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 245/486] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:03.546302 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabb050-3c50-4004-bf67-2389f2c72a4a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:03.586637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:03.633044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:03.656397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472eba83-9c1a-40b2-87da-7cdd7f3c899d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:03.694805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6a2e2-bd4b-4631-9ec9-3efc13aafb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:03.695688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6a2e2-bd4b-4631-9ec9-3efc13aafb4f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:03.696397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6a2e2-bd4b-4631-9ec9-3efc13aafb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:03.696537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6a2e2-bd4b-4631-9ec9-3efc13aafb4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:03.702531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabb050-3c50-4004-bf67-2389f2c72a4a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:03.708640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6a2e2-bd4b-4631-9ec9-3efc13aafb4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-21f6a2e2-bd4b-4631-9ec9-3efc13aafb4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:03.717168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:03.718246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:03.719034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 242/487] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:03.721295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb909d-cc29-45ef-b506-60d281951dcd None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:03.721572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb909d-cc29-45ef-b506-60d281951dcd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:03.727994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb909d-cc29-45ef-b506-60d281951dcd None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:03.727994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 243/488] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:03.727994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb909d-cc29-45ef-b506-60d281951dcd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:03.750762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f601d-1263-4da5-8069-9ae2110453bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:03.751412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f601d-1263-4da5-8069-9ae2110453bd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:03.751713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f601d-1263-4da5-8069-9ae2110453bd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:03.752139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f601d-1263-4da5-8069-9ae2110453bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:03.764577 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f601d-1263-4da5-8069-9ae2110453bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:03.766115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f601d-1263-4da5-8069-9ae2110453bd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c88f601d-1263-4da5-8069-9ae2110453bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:03.781772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:03.782700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:03.784266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:03.784266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:03.784266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:03.784266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 246/489] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:03.784664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e86aba-d072-48f6-85d7-ae4f4d4b2cd5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:03.784954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e86aba-d072-48f6-85d7-ae4f4d4b2cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:03.785250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e86aba-d072-48f6-85d7-ae4f4d4b2cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:03.816025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-81e86aba-d072-48f6-85d7-ae4f4d4b2cd5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:03.817313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-81e86aba-d072-48f6-85d7-ae4f4d4b2cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:03.847643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb909d-cc29-45ef-b506-60d281951dcd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:03.870246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:03.885853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f046c-d6b5-4976-89bb-b72182555892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:03.886449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f046c-d6b5-4976-89bb-b72182555892 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:03.886763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f046c-d6b5-4976-89bb-b72182555892 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:03.886986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f046c-d6b5-4976-89bb-b72182555892 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:03.887637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f046c-d6b5-4976-89bb-b72182555892 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:03.887937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f046c-d6b5-4976-89bb-b72182555892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:03.893536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 247/490] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:03.893943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f046c-d6b5-4976-89bb-b72182555892 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:03.894036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d48931-500a-4922-9885-4fc9598eafe8 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:03.894036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d48931-500a-4922-9885-4fc9598eafe8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:03.894036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d48931-500a-4922-9885-4fc9598eafe8 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:03.894036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 248/491] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:03.894036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d48931-500a-4922-9885-4fc9598eafe8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:03.911353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 244/492] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:03.912853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb89c1-b174-4624-888a-5a53969cad1a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:03.913138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb89c1-b174-4624-888a-5a53969cad1a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:03.913430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb89c1-b174-4624-888a-5a53969cad1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:03.924438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:03.938197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-58fb89c1-b174-4624-888a-5a53969cad1a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:03.941178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-58fb89c1-b174-4624-888a-5a53969cad1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:04.026010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d48931-500a-4922-9885-4fc9598eafe8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.033655 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 245/493] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:04.034995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb89c1-b174-4624-888a-5a53969cad1a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:04.069074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcee067-d0c8-4e85-8f53-5612a66a8370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.069774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcee067-d0c8-4e85-8f53-5612a66a8370 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.070050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcee067-d0c8-4e85-8f53-5612a66a8370 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.070317 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcee067-d0c8-4e85-8f53-5612a66a8370 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.070958 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcee067-d0c8-4e85-8f53-5612a66a8370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:04.071252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcee067-d0c8-4e85-8f53-5612a66a8370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:04.072601 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcee067-d0c8-4e85-8f53-5612a66a8370 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:04.087002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 249/494] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:04.094482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcee067-d0c8-4e85-8f53-5612a66a8370 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:04.229155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:04.232812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb89c1-b174-4624-888a-5a53969cad1a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.252377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:04.253491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcee067-d0c8-4e85-8f53-5612a66a8370 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.282548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6fcc6-ffc5-4065-9d64-7015a6ff20ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.282548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6fcc6-ffc5-4065-9d64-7015a6ff20ef None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.282548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6fcc6-ffc5-4065-9d64-7015a6ff20ef None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.282548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6fcc6-ffc5-4065-9d64-7015a6ff20ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.296476 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6fcc6-ffc5-4065-9d64-7015a6ff20ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:04.298339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6fcc6-ffc5-4065-9d64-7015a6ff20ef None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:04.305890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd18d11-abf7-40a8-a6d9-3eeb6e78af32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.306848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd18d11-abf7-40a8-a6d9-3eeb6e78af32 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.307330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd18d11-abf7-40a8-a6d9-3eeb6e78af32 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.307779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd18d11-abf7-40a8-a6d9-3eeb6e78af32 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.317942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd18d11-abf7-40a8-a6d9-3eeb6e78af32 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-eec6fcc6-ffc5-4065-9d64-7015a6ff20ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:04.319668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:04.320786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:04.321871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:04.321871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:04.321871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:04.322321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd18d11-abf7-40a8-a6d9-3eeb6e78af32 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:04.322557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 246/495] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:04.323410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6fcc6-ffc5-4065-9d64-7015a6ff20ef None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6fd18d11-abf7-40a8-a6d9-3eeb6e78af32 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:04.338660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:04.339611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:04.340543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:04.340543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:04.340543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:04.340543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:04.340543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:04.340543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:04.340543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:04.340543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:04.340543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:04.340543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:04.347466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 250/496] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:04.348570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd18d11-abf7-40a8-a6d9-3eeb6e78af32 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:04.436261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6fcc6-ffc5-4065-9d64-7015a6ff20ef None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.459375 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd18d11-abf7-40a8-a6d9-3eeb6e78af32 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.472161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f4765-57dc-40f8-9099-014f6f2b3087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.472791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f4765-57dc-40f8-9099-014f6f2b3087 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.473071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f4765-57dc-40f8-9099-014f6f2b3087 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.473356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f4765-57dc-40f8-9099-014f6f2b3087 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.484374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f4765-57dc-40f8-9099-014f6f2b3087 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-245f4765-57dc-40f8-9099-014f6f2b3087 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:04.490571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:04.491537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:04.492337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 247/497] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:04.492995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f4765-57dc-40f8-9099-014f6f2b3087 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:04.499754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d4199-7b7e-4acf-8524-8a189b17e70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.500415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4199-7b7e-4acf-8524-8a189b17e70a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.500697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4199-7b7e-4acf-8524-8a189b17e70a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.500964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4199-7b7e-4acf-8524-8a189b17e70a None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.501642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d4199-7b7e-4acf-8524-8a189b17e70a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:04.502748 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d4199-7b7e-4acf-8524-8a189b17e70a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:04.504695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d4199-7b7e-4acf-8524-8a189b17e70a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:04.521755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 251/498] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:04.522404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:04.522513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d4199-7b7e-4acf-8524-8a189b17e70a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:04.527068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:04.601585 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f4765-57dc-40f8-9099-014f6f2b3087 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.625307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d4199-7b7e-4acf-8524-8a189b17e70a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.635546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a96b0-e1ca-415b-ada0-7c2558120ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.636151 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a96b0-e1ca-415b-ada0-7c2558120ebc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.636431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a96b0-e1ca-415b-ada0-7c2558120ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.636696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a96b0-e1ca-415b-ada0-7c2558120ebc None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.637425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a96b0-e1ca-415b-ada0-7c2558120ebc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:04.637760 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a96b0-e1ca-415b-ada0-7c2558120ebc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:04.639196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a96b0-e1ca-415b-ada0-7c2558120ebc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:04.658699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c429d-e69a-40ab-a954-14d7ddb5accf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.659452 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c429d-e69a-40ab-a954-14d7ddb5accf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.659602 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c429d-e69a-40ab-a954-14d7ddb5accf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.659888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c429d-e69a-40ab-a954-14d7ddb5accf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.670146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c429d-e69a-40ab-a954-14d7ddb5accf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-b17c429d-e69a-40ab-a954-14d7ddb5accf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:04.675824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:04.677178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:04.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 252/499] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:04.678686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c429d-e69a-40ab-a954-14d7ddb5accf None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:04.685926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 248/500] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:04.685926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a96b0-e1ca-415b-ada0-7c2558120ebc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:04.744252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:04.769177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:04.783505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c429d-e69a-40ab-a954-14d7ddb5accf None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.788028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a96b0-e1ca-415b-ada0-7c2558120ebc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.816999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0448184-b137-4008-8f70-e9d3e8852f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.817637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448184-b137-4008-8f70-e9d3e8852f7d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.817924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448184-b137-4008-8f70-e9d3e8852f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.818198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448184-b137-4008-8f70-e9d3e8852f7d None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.818860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448184-b137-4008-8f70-e9d3e8852f7d None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:04.819162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448184-b137-4008-8f70-e9d3e8852f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:04.820630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448184-b137-4008-8f70-e9d3e8852f7d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:04.823219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11602c5-2c1f-43bd-afe7-f7be384429ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.824119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11602c5-2c1f-43bd-afe7-f7be384429ad None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.824393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11602c5-2c1f-43bd-afe7-f7be384429ad None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.824659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11602c5-2c1f-43bd-afe7-f7be384429ad None admin] PATH_INFO: `/v3/users/glance` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.829329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 253/501] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:04.830645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6d3a3-3000-4920-bad0-ecdc07267abf None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.830931 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6d3a3-3000-4920-bad0-ecdc07267abf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.831210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6d3a3-3000-4920-bad0-ecdc07267abf None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.832082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11602c5-2c1f-43bd-afe7-f7be384429ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:04.832563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 254/502] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:04.833176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6d3a3-3000-4920-bad0-ecdc07267abf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:04.833620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11602c5-2c1f-43bd-afe7-f7be384429ad None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-c11602c5-2c1f-43bd-afe7-f7be384429ad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:04.850358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:04.851300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:04.852436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 249/503] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:04.853207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11602c5-2c1f-43bd-afe7-f7be384429ad None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:04.945280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6d3a3-3000-4920-bad0-ecdc07267abf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.952949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11602c5-2c1f-43bd-afe7-f7be384429ad None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:04.969280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:04.979889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcf0c5-1bb7-48db-a231-c0eb631565d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.980495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcf0c5-1bb7-48db-a231-c0eb631565d0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.980769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcf0c5-1bb7-48db-a231-c0eb631565d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.981032 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcf0c5-1bb7-48db-a231-c0eb631565d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.981723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcf0c5-1bb7-48db-a231-c0eb631565d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:04.982029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcf0c5-1bb7-48db-a231-c0eb631565d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:04.983389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcf0c5-1bb7-48db-a231-c0eb631565d0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:04.987885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6531d0-66f5-4344-8d8b-01a75446c98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:04.988477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6531d0-66f5-4344-8d8b-01a75446c98e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:04.988757 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6531d0-66f5-4344-8d8b-01a75446c98e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:04.989024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6531d0-66f5-4344-8d8b-01a75446c98e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:04.996572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 255/504] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:04.996572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcf0c5-1bb7-48db-a231-c0eb631565d0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:05.006790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:05.063627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6531d0-66f5-4344-8d8b-01a75446c98e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:05.064547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6531d0-66f5-4344-8d8b-01a75446c98e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:05.092162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcf0c5-1bb7-48db-a231-c0eb631565d0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:05.127115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e070c14-dbb8-477e-8bd9-89e537d23c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:05.127663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e070c14-dbb8-477e-8bd9-89e537d23c82 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:05.127938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e070c14-dbb8-477e-8bd9-89e537d23c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:05.128204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e070c14-dbb8-477e-8bd9-89e537d23c82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:05.139030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e070c14-dbb8-477e-8bd9-89e537d23c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:05.140448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e070c14-dbb8-477e-8bd9-89e537d23c82 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7e070c14-dbb8-477e-8bd9-89e537d23c82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:05.156196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:05.157210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:05.159106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:05.159106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:05.159106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:05.159106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 256/505] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:05.159427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930495a-f9e8-4dad-acde-e811c370a2a8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:05.159709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930495a-f9e8-4dad-acde-e811c370a2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:05.159980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930495a-f9e8-4dad-acde-e811c370a2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:05.169351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 250/506] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:05.170066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6531d0-66f5-4344-8d8b-01a75446c98e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:05.177424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4930495a-f9e8-4dad-acde-e811c370a2a8 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:05.178725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4930495a-f9e8-4dad-acde-e811c370a2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:05.209191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:05.219378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:05.246214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 257/507] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:05.247033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930495a-f9e8-4dad-acde-e811c370a2a8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:05.277788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6531d0-66f5-4344-8d8b-01a75446c98e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:05.316153 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbdf1c-a170-488b-911e-9b89648e8fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:05.316755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbdf1c-a170-488b-911e-9b89648e8fae None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:05.317032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbdf1c-a170-488b-911e-9b89648e8fae None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:05.317341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbdf1c-a170-488b-911e-9b89648e8fae None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:05.318107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbdf1c-a170-488b-911e-9b89648e8fae None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:05.318414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbdf1c-a170-488b-911e-9b89648e8fae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:05.319859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbdf1c-a170-488b-911e-9b89648e8fae None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:05.337979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 251/508] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 1 05:13:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:05.338791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbdf1c-a170-488b-911e-9b89648e8fae None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:05.369030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930495a-f9e8-4dad-acde-e811c370a2a8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:05.407527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e800df7-0f68-444e-8dcf-01840ce30b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:05.408318 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e800df7-0f68-444e-8dcf-01840ce30b1d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:05.408699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e800df7-0f68-444e-8dcf-01840ce30b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:05.408971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e800df7-0f68-444e-8dcf-01840ce30b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:05.419734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e800df7-0f68-444e-8dcf-01840ce30b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:05.421196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e800df7-0f68-444e-8dcf-01840ce30b1d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0e800df7-0f68-444e-8dcf-01840ce30b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:05.437753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:05.438842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:05.440389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:05.440389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:05.440389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:05.440903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 258/509] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:05.441697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e800df7-0f68-444e-8dcf-01840ce30b1d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:05.454117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:05.458138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbdf1c-a170-488b-911e-9b89648e8fae None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:05.467013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:05.495917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8a5dd-02fc-4718-9011-2ea480883ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:05.496527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a5dd-02fc-4718-9011-2ea480883ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:05.496798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a5dd-02fc-4718-9011-2ea480883ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:05.497066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a5dd-02fc-4718-9011-2ea480883ff2 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:05.497738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8a5dd-02fc-4718-9011-2ea480883ff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:05.498038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8a5dd-02fc-4718-9011-2ea480883ff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:05.499431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8a5dd-02fc-4718-9011-2ea480883ff2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:05.511362 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 252/510] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:05.516215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8a5dd-02fc-4718-9011-2ea480883ff2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:05.551060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e800df7-0f68-444e-8dcf-01840ce30b1d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:05.589299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1df69-5772-4e66-a08a-ac5f7514e29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:05.589952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1df69-5772-4e66-a08a-ac5f7514e29e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:05.590228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1df69-5772-4e66-a08a-ac5f7514e29e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:05.590495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1df69-5772-4e66-a08a-ac5f7514e29e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:05.601536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1df69-5772-4e66-a08a-ac5f7514e29e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:05.602993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1df69-5772-4e66-a08a-ac5f7514e29e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:05.617298 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8a5dd-02fc-4718-9011-2ea480883ff2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3ae1df69-5772-4e66-a08a-ac5f7514e29e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:05.617600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:05.618447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:05.619309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:05.619309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:05.619309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:05.619309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 259/511] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:05.620049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1df69-5772-4e66-a08a-ac5f7514e29e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:05.651744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd164a27-616a-4a5a-8964-026209568afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:05.652349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd164a27-616a-4a5a-8964-026209568afb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:05.652624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd164a27-616a-4a5a-8964-026209568afb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:05.652904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd164a27-616a-4a5a-8964-026209568afb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:05.681537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:05.691345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:05.721343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1df69-5772-4e66-a08a-ac5f7514e29e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:05.726182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd164a27-616a-4a5a-8964-026209568afb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:05.727861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd164a27-616a-4a5a-8964-026209568afb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:05.756533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cd766-f694-4311-84a8-06be5e4c2b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:05.757295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cd766-f694-4311-84a8-06be5e4c2b0c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:05.757717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cd766-f694-4311-84a8-06be5e4c2b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:05.758086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cd766-f694-4311-84a8-06be5e4c2b0c None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:05.758821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cd766-f694-4311-84a8-06be5e4c2b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:05.759235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cd766-f694-4311-84a8-06be5e4c2b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:05.760655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cd766-f694-4311-84a8-06be5e4c2b0c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:05.772232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 260/512] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:05.773024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cd766-f694-4311-84a8-06be5e4c2b0c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:05.833799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 253/513] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:05.834765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd164a27-616a-4a5a-8964-026209568afb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:05.877341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cd766-f694-4311-84a8-06be5e4c2b0c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:05.893169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:05.907073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:05.912433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817051b0-9e23-4f1a-9f47-7821c3ef8096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:05.913060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817051b0-9e23-4f1a-9f47-7821c3ef8096 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:05.913334 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817051b0-9e23-4f1a-9f47-7821c3ef8096 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:05.913634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817051b0-9e23-4f1a-9f47-7821c3ef8096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:05.923801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817051b0-9e23-4f1a-9f47-7821c3ef8096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:05.925221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817051b0-9e23-4f1a-9f47-7821c3ef8096 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:05.933539 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd164a27-616a-4a5a-8964-026209568afb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-817051b0-9e23-4f1a-9f47-7821c3ef8096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:05.939724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:05.940805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:05.942187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:05.942187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:05.942187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:05.945771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 261/514] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:05.946867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817051b0-9e23-4f1a-9f47-7821c3ef8096 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:05.967605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0ffc6-4d57-4d73-98bc-d94c51919f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:05.968220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0ffc6-4d57-4d73-98bc-d94c51919f24 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:05.968494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0ffc6-4d57-4d73-98bc-d94c51919f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:05.968766 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0ffc6-4d57-4d73-98bc-d94c51919f24 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:05.977579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0ffc6-4d57-4d73-98bc-d94c51919f24 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:05.979054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0ffc6-4d57-4d73-98bc-d94c51919f24 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-afe0ffc6-4d57-4d73-98bc-d94c51919f24 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:05.990236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:05.991264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 254/515] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:05.996937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0ffc6-4d57-4d73-98bc-d94c51919f24 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.046292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817051b0-9e23-4f1a-9f47-7821c3ef8096 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.080328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38305991-de6f-4617-9126-ce36fe709e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.080938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38305991-de6f-4617-9126-ce36fe709e33 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.081219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38305991-de6f-4617-9126-ce36fe709e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.081520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38305991-de6f-4617-9126-ce36fe709e33 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.082160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38305991-de6f-4617-9126-ce36fe709e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:06.082466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38305991-de6f-4617-9126-ce36fe709e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.083844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38305991-de6f-4617-9126-ce36fe709e33 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.095140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0ffc6-4d57-4d73-98bc-d94c51919f24 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.096872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 262/516] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:06.097729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38305991-de6f-4617-9126-ce36fe709e33 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.119691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:06.133922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ad892-626a-4378-966a-a72f342ccc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.134547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ad892-626a-4378-966a-a72f342ccc45 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.134828 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ad892-626a-4378-966a-a72f342ccc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.135102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ad892-626a-4378-966a-a72f342ccc45 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.141167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:06.143012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ad892-626a-4378-966a-a72f342ccc45 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.144455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ad892-626a-4378-966a-a72f342ccc45 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-223ad892-626a-4378-966a-a72f342ccc45 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:06.155348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:06.156457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 255/517] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:06.157351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ad892-626a-4378-966a-a72f342ccc45 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.199040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38305991-de6f-4617-9126-ce36fe709e33 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.232651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48030f40-0aa8-4c89-aaff-9586809fee87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.233280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48030f40-0aa8-4c89-aaff-9586809fee87 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.233606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48030f40-0aa8-4c89-aaff-9586809fee87 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.233882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48030f40-0aa8-4c89-aaff-9586809fee87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.244341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48030f40-0aa8-4c89-aaff-9586809fee87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.245826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48030f40-0aa8-4c89-aaff-9586809fee87 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.259385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ad892-626a-4378-966a-a72f342ccc45 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-48030f40-0aa8-4c89-aaff-9586809fee87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:06.260603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:06.261657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:06.262526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:06.262526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:06.262526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:06.262526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 263/518] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:06.262781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48030f40-0aa8-4c89-aaff-9586809fee87 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.295279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295816f-b6ea-47be-adfb-97682c9dd08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.295880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295816f-b6ea-47be-adfb-97682c9dd08b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.296161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295816f-b6ea-47be-adfb-97682c9dd08b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.296430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295816f-b6ea-47be-adfb-97682c9dd08b None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.297071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295816f-b6ea-47be-adfb-97682c9dd08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:06.297376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295816f-b6ea-47be-adfb-97682c9dd08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.299438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295816f-b6ea-47be-adfb-97682c9dd08b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.310853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 256/519] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:06.311636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295816f-b6ea-47be-adfb-97682c9dd08b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.353143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:06.356998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:06.363013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48030f40-0aa8-4c89-aaff-9586809fee87 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.395252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ad4b0-495f-493b-b004-36cefa3258f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.395848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ad4b0-495f-493b-b004-36cefa3258f0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.396128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ad4b0-495f-493b-b004-36cefa3258f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.396391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ad4b0-495f-493b-b004-36cefa3258f0 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.397106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ad4b0-495f-493b-b004-36cefa3258f0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:06.397596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ad4b0-495f-493b-b004-36cefa3258f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.399028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ad4b0-495f-493b-b004-36cefa3258f0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.410981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295816f-b6ea-47be-adfb-97682c9dd08b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.435689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 264/520] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 1 05:13:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:06.436630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ad4b0-495f-493b-b004-36cefa3258f0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.450057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3f7d-c66e-4900-b1d8-dad421115286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.450619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3f7d-c66e-4900-b1d8-dad421115286 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.450899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3f7d-c66e-4900-b1d8-dad421115286 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.451172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3f7d-c66e-4900-b1d8-dad421115286 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.457911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f3f7d-c66e-4900-b1d8-dad421115286 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.459360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f3f7d-c66e-4900-b1d8-dad421115286 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-b32f3f7d-c66e-4900-b1d8-dad421115286 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:06.471707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:06.472594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:06.473507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:06.473507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:06.473507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:06.473507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:06.473507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:06.473507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:06.473507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:06.473507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:06.473507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 257/521] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:06.474124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3f7d-c66e-4900-b1d8-dad421115286 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.537789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ad4b0-495f-493b-b004-36cefa3258f0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.570260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:06.571404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f09c74-80d2-4f27-9f78-f265d0891fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.572015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f09c74-80d2-4f27-9f78-f265d0891fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.572297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f09c74-80d2-4f27-9f78-f265d0891fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.572570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f09c74-80d2-4f27-9f78-f265d0891fa5 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.573103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3f7d-c66e-4900-b1d8-dad421115286 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.573291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f09c74-80d2-4f27-9f78-f265d0891fa5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:06.573629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f09c74-80d2-4f27-9f78-f265d0891fa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.575327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f09c74-80d2-4f27-9f78-f265d0891fa5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.592321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 265/522] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Sep 1 05:13:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:06.593110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f09c74-80d2-4f27-9f78-f265d0891fa5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.608030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c786f-b96c-4ccf-8192-652225a367a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.608837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c786f-b96c-4ccf-8192-652225a367a2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.609323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c786f-b96c-4ccf-8192-652225a367a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.609448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:06.609997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c786f-b96c-4ccf-8192-652225a367a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.610674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c786f-b96c-4ccf-8192-652225a367a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:06.611530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c786f-b96c-4ccf-8192-652225a367a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.613234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c786f-b96c-4ccf-8192-652225a367a2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.624863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 258/523] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:06.625551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c786f-b96c-4ccf-8192-652225a367a2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.694686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f09c74-80d2-4f27-9f78-f265d0891fa5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.728666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c786f-b96c-4ccf-8192-652225a367a2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.731306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadab17-24a5-4192-9f9d-4057f1f1b7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.731923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadab17-24a5-4192-9f9d-4057f1f1b7fb None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.732203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadab17-24a5-4192-9f9d-4057f1f1b7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.732477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadab17-24a5-4192-9f9d-4057f1f1b7fb None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.733122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadab17-24a5-4192-9f9d-4057f1f1b7fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.734591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadab17-24a5-4192-9f9d-4057f1f1b7fb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.766292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295ae06-8088-4251-a294-a2f4921653e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.766899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295ae06-8088-4251-a294-a2f4921653e3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.767698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295ae06-8088-4251-a294-a2f4921653e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.768158 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295ae06-8088-4251-a294-a2f4921653e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.796801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:06.821500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 266/524] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:13:06 2022] POST /identity/v3/users => generated 247 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:06.821500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadab17-24a5-4192-9f9d-4057f1f1b7fb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.840677 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295ae06-8088-4251-a294-a2f4921653e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.842277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295ae06-8088-4251-a294-a2f4921653e3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.889187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:06.928022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadab17-24a5-4192-9f9d-4057f1f1b7fb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:06.960464 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 259/525] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:06.961494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295ae06-8088-4251-a294-a2f4921653e3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:06.968142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007d0b7-d847-4af8-8902-a7b047064898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:06.968792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007d0b7-d847-4af8-8902-a7b047064898 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:06.969070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007d0b7-d847-4af8-8902-a7b047064898 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:06.969343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007d0b7-d847-4af8-8902-a7b047064898 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:06.970095 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007d0b7-d847-4af8-8902-a7b047064898 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:06.970585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007d0b7-d847-4af8-8902-a7b047064898 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:06.972238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007d0b7-d847-4af8-8902-a7b047064898 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:06.991345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 267/526] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:06.991971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007d0b7-d847-4af8-8902-a7b047064898 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:07.033438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:07.080376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295ae06-8088-4251-a294-a2f4921653e3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:07.120584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3e4cf-a4a6-4ca1-ad65-34ed673fa27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:07.121167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007d0b7-d847-4af8-8902-a7b047064898 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:07.121311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3e4cf-a4a6-4ca1-ad65-34ed673fa27b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:07.121561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3e4cf-a4a6-4ca1-ad65-34ed673fa27b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:07.121768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3e4cf-a4a6-4ca1-ad65-34ed673fa27b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:07.154540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:07.164602 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0fd0-0c9e-4e27-8f5a-49875d2c60d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:07.166060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0fd0-0c9e-4e27-8f5a-49875d2c60d8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:07.166060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0fd0-0c9e-4e27-8f5a-49875d2c60d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:07.166294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0fd0-0c9e-4e27-8f5a-49875d2c60d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:07.175013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b0fd0-0c9e-4e27-8f5a-49875d2c60d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:07.177023 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b0fd0-0c9e-4e27-8f5a-49875d2c60d8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-122b0fd0-0c9e-4e27-8f5a-49875d2c60d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:07.191218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:07.192174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:07.193029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:07.193029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:07.193029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:07.193029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:07.193029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:07.193029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:07.193029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:07.193029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:07.193029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 268/527] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:07.194091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0fd0-0c9e-4e27-8f5a-49875d2c60d8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:07.201591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3e4cf-a4a6-4ca1-ad65-34ed673fa27b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:07.211917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3e4cf-a4a6-4ca1-ad65-34ed673fa27b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:07.269487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:07.302941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0fd0-0c9e-4e27-8f5a-49875d2c60d8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:07.323324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 260/528] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:07.323324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3e4cf-a4a6-4ca1-ad65-34ed673fa27b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:07.351123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a055d-0f92-4c3c-8d07-9e7f56628901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:07.351860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a055d-0f92-4c3c-8d07-9e7f56628901 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:07.352138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a055d-0f92-4c3c-8d07-9e7f56628901 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:07.352412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a055d-0f92-4c3c-8d07-9e7f56628901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:07.363337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a055d-0f92-4c3c-8d07-9e7f56628901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:07.364836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a055d-0f92-4c3c-8d07-9e7f56628901 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-429a055d-0f92-4c3c-8d07-9e7f56628901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:07.379541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:07.380652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:07.381727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:07.381727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:07.381727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:07.381727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 269/529] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:07.385780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a055d-0f92-4c3c-8d07-9e7f56628901 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:07.431562 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3e4cf-a4a6-4ca1-ad65-34ed673fa27b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:07.440192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:07.473220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e6544-a488-4216-96ab-f6c60b94673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:07.473996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6544-a488-4216-96ab-f6c60b94673a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:07.474375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6544-a488-4216-96ab-f6c60b94673a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:07.474746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6544-a488-4216-96ab-f6c60b94673a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:07.475574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e6544-a488-4216-96ab-f6c60b94673a None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=7cda3c1f67e54ee08abc014399b07cc5) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:07.476191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e6544-a488-4216-96ab-f6c60b94673a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:07.477529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e6544-a488-4216-96ab-f6c60b94673a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:07.489491 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 261/530] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:07 2022] GET /identity/v3/role_assignments?user.id=7cda3c1f67e54ee08abc014399b07cc5&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:07.489491 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e6544-a488-4216-96ab-f6c60b94673a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:07.519040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a055d-0f92-4c3c-8d07-9e7f56628901 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:07.523580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:07.563385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18d632-5628-4884-97bb-d4c284c8b3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:07.563840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d632-5628-4884-97bb-d4c284c8b3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:07.564869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d632-5628-4884-97bb-d4c284c8b3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:07.565214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d632-5628-4884-97bb-d4c284c8b3f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:07.566053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18d632-5628-4884-97bb-d4c284c8b3f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:07.566491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18d632-5628-4884-97bb-d4c284c8b3f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:07.568008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18d632-5628-4884-97bb-d4c284c8b3f7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:07.584119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 270/531] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:07.585121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18d632-5628-4884-97bb-d4c284c8b3f7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:07.625362 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e6544-a488-4216-96ab-f6c60b94673a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:07.668165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfcb88-4f6c-409b-b81d-1b7bd16b62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:07.669088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfcb88-4f6c-409b-b81d-1b7bd16b62e6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:07.669368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfcb88-4f6c-409b-b81d-1b7bd16b62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:07.669684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfcb88-4f6c-409b-b81d-1b7bd16b62e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:07.682819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfcb88-4f6c-409b-b81d-1b7bd16b62e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:07.684454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfcb88-4f6c-409b-b81d-1b7bd16b62e6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-77cfcb88-4f6c-409b-b81d-1b7bd16b62e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:07.703866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:07.705072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:07.706323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:07.706323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:07.706323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:07.706323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 262/532] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:07.706323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfcb88-4f6c-409b-b81d-1b7bd16b62e6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:07.734245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18d632-5628-4884-97bb-d4c284c8b3f7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:07.745483 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:07.780853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6dcfc-df5b-4cc7-880e-b8f873d07e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:07.782012 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6dcfc-df5b-4cc7-880e-b8f873d07e78 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:07.783778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6dcfc-df5b-4cc7-880e-b8f873d07e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:07.784537 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6dcfc-df5b-4cc7-880e-b8f873d07e78 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:07.785553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6dcfc-df5b-4cc7-880e-b8f873d07e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:07.786221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6dcfc-df5b-4cc7-880e-b8f873d07e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:07.788321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6dcfc-df5b-4cc7-880e-b8f873d07e78 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:07.803810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 271/533] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:07.804926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6dcfc-df5b-4cc7-880e-b8f873d07e78 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:07.812527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:07.839066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfcb88-4f6c-409b-b81d-1b7bd16b62e6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:07.875222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639a282-1a38-43d7-9c80-ca799a11fd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:07.876095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639a282-1a38-43d7-9c80-ca799a11fd74 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:07.877208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639a282-1a38-43d7-9c80-ca799a11fd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:07.877925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639a282-1a38-43d7-9c80-ca799a11fd74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:07.878842 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639a282-1a38-43d7-9c80-ca799a11fd74 None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=4c23597e0da2454abd2a5354d779b092) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:07.879308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639a282-1a38-43d7-9c80-ca799a11fd74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:07.880887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639a282-1a38-43d7-9c80-ca799a11fd74 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:07.893934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 263/534] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:07 2022] GET /identity/v3/role_assignments?user.id=4c23597e0da2454abd2a5354d779b092&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:07.895155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639a282-1a38-43d7-9c80-ca799a11fd74 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:07.920782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6dcfc-df5b-4cc7-880e-b8f873d07e78 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:07.963974 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75f350-4d57-42d9-aa6e-5a6309dfefee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:07.964769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75f350-4d57-42d9-aa6e-5a6309dfefee None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:07.965165 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75f350-4d57-42d9-aa6e-5a6309dfefee None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:07.965570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75f350-4d57-42d9-aa6e-5a6309dfefee None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:07.966514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75f350-4d57-42d9-aa6e-5a6309dfefee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:07.966955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75f350-4d57-42d9-aa6e-5a6309dfefee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:07.969062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75f350-4d57-42d9-aa6e-5a6309dfefee None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:07.986797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 272/535] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:08.000693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75f350-4d57-42d9-aa6e-5a6309dfefee None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:08.041949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:08.109542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639a282-1a38-43d7-9c80-ca799a11fd74 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:08.150630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f61ba-ae7c-4e9f-a623-57a5c37442ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:08.151422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f61ba-ae7c-4e9f-a623-57a5c37442ea None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:08.151931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f61ba-ae7c-4e9f-a623-57a5c37442ea None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:08.152432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f61ba-ae7c-4e9f-a623-57a5c37442ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:08.160605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f61ba-ae7c-4e9f-a623-57a5c37442ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:08.162225 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f61ba-ae7c-4e9f-a623-57a5c37442ea None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:08.169781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-4a9f61ba-ae7c-4e9f-a623-57a5c37442ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:08.175859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:08.176960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:08.178023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:08.178023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:08.178023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:08.178023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:08.178023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:08.178023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:08.178023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:08.178023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:08.179340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 264/536] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:08.184765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f61ba-ae7c-4e9f-a623-57a5c37442ea None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:08.212139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75f350-4d57-42d9-aa6e-5a6309dfefee None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:08.296196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f61ba-ae7c-4e9f-a623-57a5c37442ea None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:08.330872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb58709-a196-49a8-9069-4885654567dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:08.331797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb58709-a196-49a8-9069-4885654567dd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:08.332077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb58709-a196-49a8-9069-4885654567dd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:08.332343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb58709-a196-49a8-9069-4885654567dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:08.332856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e0825-96e1-4bda-97b5-9251728d4c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:08.333482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e0825-96e1-4bda-97b5-9251728d4c82 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:08.333785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e0825-96e1-4bda-97b5-9251728d4c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:08.334058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e0825-96e1-4bda-97b5-9251728d4c82 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:08.334941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e0825-96e1-4bda-97b5-9251728d4c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:08.335248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e0825-96e1-4bda-97b5-9251728d4c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:08.336687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e0825-96e1-4bda-97b5-9251728d4c82 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:08.350569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb58709-a196-49a8-9069-4885654567dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:08.352168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb58709-a196-49a8-9069-4885654567dd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:08.359648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 265/537] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:08.366318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e0825-96e1-4bda-97b5-9251728d4c82 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-2bb58709-a196-49a8-9069-4885654567dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:08.371773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:08.372616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:08.373474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:08.373474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:08.373474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:08.373474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:08.373474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:08.373474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:08.373474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:08.373474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:08.375773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 273/538] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:08.381545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb58709-a196-49a8-9069-4885654567dd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:08.488795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:08.533338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:08.569123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e0825-96e1-4bda-97b5-9251728d4c82 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:08.572407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb58709-a196-49a8-9069-4885654567dd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:08.630366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118cd4b-3e08-474b-8bb0-d046c53e2a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:08.630978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118cd4b-3e08-474b-8bb0-d046c53e2a17 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:08.631253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118cd4b-3e08-474b-8bb0-d046c53e2a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:08.631521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118cd4b-3e08-474b-8bb0-d046c53e2a17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:08.632252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118cd4b-3e08-474b-8bb0-d046c53e2a17 None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=7e0e8589931b46ca84168a2ad273b63d) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:08.632559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118cd4b-3e08-474b-8bb0-d046c53e2a17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:08.634043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118cd4b-3e08-474b-8bb0-d046c53e2a17 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:08.644675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 266/539] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:08 2022] GET /identity/v3/role_assignments?user.id=7e0e8589931b46ca84168a2ad273b63d&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:08.655790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d7c37-8a42-4455-97e7-cb806e4b4684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:08.656423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d7c37-8a42-4455-97e7-cb806e4b4684 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:08.656700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d7c37-8a42-4455-97e7-cb806e4b4684 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:08.656973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d7c37-8a42-4455-97e7-cb806e4b4684 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:08.658243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d7c37-8a42-4455-97e7-cb806e4b4684 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:08.658668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d7c37-8a42-4455-97e7-cb806e4b4684 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:08.659996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d7c37-8a42-4455-97e7-cb806e4b4684 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:08.680704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 274/540] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:08.686887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118cd4b-3e08-474b-8bb0-d046c53e2a17 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:08.812304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003ee44-2e2f-4a39-b4d1-a8033a521439 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:08.812870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003ee44-2e2f-4a39-b4d1-a8033a521439 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:08.812870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003ee44-2e2f-4a39-b4d1-a8033a521439 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:08.815483 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 275/541] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:13:08.821253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59649ac-ff74-4e4c-ba4c-72a28b1afd84 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:08.821826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59649ac-ff74-4e4c-ba4c-72a28b1afd84 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:08.822337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59649ac-ff74-4e4c-ba4c-72a28b1afd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:08.844523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d59649ac-ff74-4e4c-ba4c-72a28b1afd84 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:08.846324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d59649ac-ff74-4e4c-ba4c-72a28b1afd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:08.874733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118cd4b-3e08-474b-8bb0-d046c53e2a17 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:08.900941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:08.917192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff672c1f-80de-4f2e-859e-0cb53840bf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:08.917885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff672c1f-80de-4f2e-859e-0cb53840bf18 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:08.918162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff672c1f-80de-4f2e-859e-0cb53840bf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:08.918433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff672c1f-80de-4f2e-859e-0cb53840bf18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:08.919159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff672c1f-80de-4f2e-859e-0cb53840bf18 None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=0e21910fca40479ca04e7f0418f7dd2d) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:08.919465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff672c1f-80de-4f2e-859e-0cb53840bf18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:08.920898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff672c1f-80de-4f2e-859e-0cb53840bf18 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:08.935174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 267/542] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:08 2022] GET /identity/v3/role_assignments?user.id=0e21910fca40479ca04e7f0418f7dd2d&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:08.991383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 276/543] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:13:09.371057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff672c1f-80de-4f2e-859e-0cb53840bf18 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:09.510265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff672c1f-80de-4f2e-859e-0cb53840bf18 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:09.559030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b1a2-3e7c-48de-b76b-dce013e39dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:09.559665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b1a2-3e7c-48de-b76b-dce013e39dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:09.559949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b1a2-3e7c-48de-b76b-dce013e39dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:09.560222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b1a2-3e7c-48de-b76b-dce013e39dc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:09.575637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1b1a2-3e7c-48de-b76b-dce013e39dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-24d1b1a2-3e7c-48de-b76b-dce013e39dc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:09.581510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:09.582428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:09.583882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 268/544] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:09.588424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59649ac-ff74-4e4c-ba4c-72a28b1afd84 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:09.698038 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:09.747893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59649ac-ff74-4e4c-ba4c-72a28b1afd84 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:09.788535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc0142-8069-42a5-81e8-2c4a706c30e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:09.789209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc0142-8069-42a5-81e8-2c4a706c30e8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:09.789507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc0142-8069-42a5-81e8-2c4a706c30e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:09.789779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc0142-8069-42a5-81e8-2c4a706c30e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:09.790436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc0142-8069-42a5-81e8-2c4a706c30e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:09.790735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc0142-8069-42a5-81e8-2c4a706c30e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:09.792084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc0142-8069-42a5-81e8-2c4a706c30e8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:09.804899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 277/545] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:09.810345 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b1a2-3e7c-48de-b76b-dce013e39dc5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:09.897552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:09.972648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e4188-0abc-4ad2-bce3-9a39cfe43933 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:09.973154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e4188-0abc-4ad2-bce3-9a39cfe43933 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:09.973154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e4188-0abc-4ad2-bce3-9a39cfe43933 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:09.974821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 278/546] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:13:09.975026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b1a2-3e7c-48de-b76b-dce013e39dc5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:09.979982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7da9f-047a-4e82-b291-2954358c9621 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:09.980258 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7da9f-047a-4e82-b291-2954358c9621 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:09.980527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7da9f-047a-4e82-b291-2954358c9621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:10.014332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0cd7da9f-047a-4e82-b291-2954358c9621 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:10.015519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0cd7da9f-047a-4e82-b291-2954358c9621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:10.054966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221e506-a339-425a-a2d9-87585656ecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:10.055599 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221e506-a339-425a-a2d9-87585656ecd5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:10.055873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221e506-a339-425a-a2d9-87585656ecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:10.056141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221e506-a339-425a-a2d9-87585656ecd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:10.074796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221e506-a339-425a-a2d9-87585656ecd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:10.076247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221e506-a339-425a-a2d9-87585656ecd5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e221e506-a339-425a-a2d9-87585656ecd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:10.102621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:10.103665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:10.104501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:10.104501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:10.104501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:10.106284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 269/547] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:10.109617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221e506-a339-425a-a2d9-87585656ecd5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:10.139606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 279/548] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:13:10.156613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d9d4b-2cd3-4cce-a4ff-f3609d720054 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:10.156894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d9d4b-2cd3-4cce-a4ff-f3609d720054 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:10.157167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d9d4b-2cd3-4cce-a4ff-f3609d720054 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:10.159404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 280/549] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:10.164838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61856673-5891-411b-86ad-ffb5f3ee13e9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:10.165120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61856673-5891-411b-86ad-ffb5f3ee13e9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:10.165472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61856673-5891-411b-86ad-ffb5f3ee13e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:10.196413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-61856673-5891-411b-86ad-ffb5f3ee13e9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:10.199454 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-61856673-5891-411b-86ad-ffb5f3ee13e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:10.294116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:10.296632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:10.302547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221e506-a339-425a-a2d9-87585656ecd5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:10.313476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 281/550] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:13:10.359897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6b4b3-c0ed-4b7c-ac1d-3aacc7ab526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:10.360932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6b4b3-c0ed-4b7c-ac1d-3aacc7ab526f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:10.361648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6b4b3-c0ed-4b7c-ac1d-3aacc7ab526f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:10.362071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6b4b3-c0ed-4b7c-ac1d-3aacc7ab526f None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:10.362869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6b4b3-c0ed-4b7c-ac1d-3aacc7ab526f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:10.363313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6b4b3-c0ed-4b7c-ac1d-3aacc7ab526f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:10.364887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6b4b3-c0ed-4b7c-ac1d-3aacc7ab526f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:10.378040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 270/551] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:10.383737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61856673-5891-411b-86ad-ffb5f3ee13e9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:10.459227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6b4b3-c0ed-4b7c-ac1d-3aacc7ab526f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:10.537603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61856673-5891-411b-86ad-ffb5f3ee13e9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:10.575960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f185309-6872-49db-ae2f-f0a91fd3d8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:10.576775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185309-6872-49db-ae2f-f0a91fd3d8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:10.577056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185309-6872-49db-ae2f-f0a91fd3d8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:10.577332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185309-6872-49db-ae2f-f0a91fd3d8f3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:10.586176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f185309-6872-49db-ae2f-f0a91fd3d8f3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:10.587630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f185309-6872-49db-ae2f-f0a91fd3d8f3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1f185309-6872-49db-ae2f-f0a91fd3d8f3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:10.601046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:10.602026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:10.602953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 282/552] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:10.609815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f185309-6872-49db-ae2f-f0a91fd3d8f3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:10.667659 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:10.694714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6b4b3-c0ed-4b7c-ac1d-3aacc7ab526f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:10.730685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f185309-6872-49db-ae2f-f0a91fd3d8f3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:10.743509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313b362-42f2-430e-a256-e843f2997b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:10.744383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313b362-42f2-430e-a256-e843f2997b6a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:10.744837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313b362-42f2-430e-a256-e843f2997b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:10.745286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313b362-42f2-430e-a256-e843f2997b6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:10.756720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313b362-42f2-430e-a256-e843f2997b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-2313b362-42f2-430e-a256-e843f2997b6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:10.762177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:10.763075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:10.763901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:10.771469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 271/553] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:10.771469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313b362-42f2-430e-a256-e843f2997b6a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:10.777517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563548-46b8-4963-87da-7b8ead1dd573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:10.778146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563548-46b8-4963-87da-7b8ead1dd573 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:10.778421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563548-46b8-4963-87da-7b8ead1dd573 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:10.778697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563548-46b8-4963-87da-7b8ead1dd573 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:10.779440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e563548-46b8-4963-87da-7b8ead1dd573 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:10.779744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e563548-46b8-4963-87da-7b8ead1dd573 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:10.781183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e563548-46b8-4963-87da-7b8ead1dd573 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:10.795678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:10.805169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 283/554] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 1 05:13:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:10.806292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563548-46b8-4963-87da-7b8ead1dd573 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:10.881739 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313b362-42f2-430e-a256-e843f2997b6a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:10.925523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec57aa-f82f-4566-a94c-ffd1b4e6ddae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:10.926730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec57aa-f82f-4566-a94c-ffd1b4e6ddae None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:10.927015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec57aa-f82f-4566-a94c-ffd1b4e6ddae None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:10.927440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec57aa-f82f-4566-a94c-ffd1b4e6ddae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:10.938342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec57aa-f82f-4566-a94c-ffd1b4e6ddae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:10.940928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563548-46b8-4963-87da-7b8ead1dd573 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8cec57aa-f82f-4566-a94c-ffd1b4e6ddae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:10.943719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:10.944658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 272/555] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:10.945802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec57aa-f82f-4566-a94c-ffd1b4e6ddae None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:10.958821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:10.982511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b21641-cc91-43ad-a9b9-f871ee191052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:10.983323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21641-cc91-43ad-a9b9-f871ee191052 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:10.983721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21641-cc91-43ad-a9b9-f871ee191052 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:10.984057 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21641-cc91-43ad-a9b9-f871ee191052 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:10.984721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b21641-cc91-43ad-a9b9-f871ee191052 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:10.985023 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b21641-cc91-43ad-a9b9-f871ee191052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:10.986471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b21641-cc91-43ad-a9b9-f871ee191052 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:10.996196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 284/556] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:10.998202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5766cb-d2e5-4763-8778-8c6cd8bd5567 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.000980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5766cb-d2e5-4763-8778-8c6cd8bd5567 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.000980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5766cb-d2e5-4763-8778-8c6cd8bd5567 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.000980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 285/557] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:11.002176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5766cb-d2e5-4763-8778-8c6cd8bd5567 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:11.043473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:11.063688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec57aa-f82f-4566-a94c-ffd1b4e6ddae None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:11.101728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438cfe5-fc99-4202-87bd-3b8c1eb869cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:11.102375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438cfe5-fc99-4202-87bd-3b8c1eb869cf None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.102604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438cfe5-fc99-4202-87bd-3b8c1eb869cf None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.102870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438cfe5-fc99-4202-87bd-3b8c1eb869cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.114458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5766cb-d2e5-4763-8778-8c6cd8bd5567 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:11.152176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687bf28-ec26-4ac7-b3d8-c22f80901905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:11.152711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687bf28-ec26-4ac7-b3d8-c22f80901905 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.152994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687bf28-ec26-4ac7-b3d8-c22f80901905 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.153271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687bf28-ec26-4ac7-b3d8-c22f80901905 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.153962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687bf28-ec26-4ac7-b3d8-c22f80901905 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:11.154518 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687bf28-ec26-4ac7-b3d8-c22f80901905 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:11.155919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687bf28-ec26-4ac7-b3d8-c22f80901905 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:11.168964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 286/558] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:11.168964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687bf28-ec26-4ac7-b3d8-c22f80901905 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:11.194584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438cfe5-fc99-4202-87bd-3b8c1eb869cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:11.196120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438cfe5-fc99-4202-87bd-3b8c1eb869cf None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:11.203165 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:11.275681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687bf28-ec26-4ac7-b3d8-c22f80901905 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:11.294797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:11.314079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bf17c-f886-4a06-b522-4f1f93d14cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:11.314723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bf17c-f886-4a06-b522-4f1f93d14cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.315000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bf17c-f886-4a06-b522-4f1f93d14cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.319614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bf17c-f886-4a06-b522-4f1f93d14cd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.319743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 273/559] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:11.319743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54e6ff-9107-4f23-8ba0-c9883a654cac None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.319743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54e6ff-9107-4f23-8ba0-c9883a654cac None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.319743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54e6ff-9107-4f23-8ba0-c9883a654cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.330941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bf17c-f886-4a06-b522-4f1f93d14cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:11.332436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bf17c-f886-4a06-b522-4f1f93d14cd9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:11.337089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ef54e6ff-9107-4f23-8ba0-c9883a654cac None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:11.338494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ef54e6ff-9107-4f23-8ba0-c9883a654cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-bf3bf17c-f886-4a06-b522-4f1f93d14cd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:11.349168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:11.350189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:11.351113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:11.351113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:11.351113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:11.353798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 287/560] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:11.354651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bf17c-f886-4a06-b522-4f1f93d14cd9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:11.412384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 274/561] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:11.413140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54e6ff-9107-4f23-8ba0-c9883a654cac None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:11.444916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:11.461564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bf17c-f886-4a06-b522-4f1f93d14cd9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:11.498441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ccab2-20b0-480d-af7a-e07a92fb590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:11.499069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ccab2-20b0-480d-af7a-e07a92fb590b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.499344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ccab2-20b0-480d-af7a-e07a92fb590b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.499610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ccab2-20b0-480d-af7a-e07a92fb590b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.511216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ccab2-20b0-480d-af7a-e07a92fb590b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:11.512669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ccab2-20b0-480d-af7a-e07a92fb590b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:11.513533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54e6ff-9107-4f23-8ba0-c9883a654cac None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:11.526606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f46ccab2-20b0-480d-af7a-e07a92fb590b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:11.527745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:11.528686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:11.529620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:11.529620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:11.529620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:11.529620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 288/562] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:11.530223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ccab2-20b0-480d-af7a-e07a92fb590b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:11.557106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5951c0-c194-4173-a7a2-3cab5d513598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:11.558162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5951c0-c194-4173-a7a2-3cab5d513598 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.558650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5951c0-c194-4173-a7a2-3cab5d513598 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.559367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5951c0-c194-4173-a7a2-3cab5d513598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.571322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5951c0-c194-4173-a7a2-3cab5d513598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:11.572782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5951c0-c194-4173-a7a2-3cab5d513598 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-9f5951c0-c194-4173-a7a2-3cab5d513598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:11.588374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:11.589434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:11.590374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:11.590374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:11.590374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:11.590374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 275/563] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:11.591288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb46a4-38b6-455c-8d80-182580b81aa4 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.591288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb46a4-38b6-455c-8d80-182580b81aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.591679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb46a4-38b6-455c-8d80-182580b81aa4 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.593140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 276/564] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:11.597835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb46a4-38b6-455c-8d80-182580b81aa4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:11.656503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ccab2-20b0-480d-af7a-e07a92fb590b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:11.695350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1076f-f4d4-4b4f-96e8-2a08c74e5cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:11.696183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1076f-f4d4-4b4f-96e8-2a08c74e5cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.696467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1076f-f4d4-4b4f-96e8-2a08c74e5cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.696741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1076f-f4d4-4b4f-96e8-2a08c74e5cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.697414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1076f-f4d4-4b4f-96e8-2a08c74e5cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:11.697747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1076f-f4d4-4b4f-96e8-2a08c74e5cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:11.699421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1076f-f4d4-4b4f-96e8-2a08c74e5cb5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:11.707962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:11.713975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 289/565] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:11.714407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1076f-f4d4-4b4f-96e8-2a08c74e5cb5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:11.720216 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb46a4-38b6-455c-8d80-182580b81aa4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:11.757816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f41d43-4a54-4658-aa14-3cbfba2e41ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:11.758414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41d43-4a54-4658-aa14-3cbfba2e41ea None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.758707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41d43-4a54-4658-aa14-3cbfba2e41ea None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.758955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f41d43-4a54-4658-aa14-3cbfba2e41ea None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.759587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41d43-4a54-4658-aa14-3cbfba2e41ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:11.759885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41d43-4a54-4658-aa14-3cbfba2e41ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:11.761238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f41d43-4a54-4658-aa14-3cbfba2e41ea None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:11.773690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 277/566] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:11.774828 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707c2a3-12c3-47f4-85a2-d08c3d9303ce None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.775103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707c2a3-12c3-47f4-85a2-d08c3d9303ce None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.775366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707c2a3-12c3-47f4-85a2-d08c3d9303ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.793858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c707c2a3-12c3-47f4-85a2-d08c3d9303ce None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:11.795036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c707c2a3-12c3-47f4-85a2-d08c3d9303ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:11.811564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:11.819064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1076f-f4d4-4b4f-96e8-2a08c74e5cb5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:11.855996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c5192-1db9-431c-bb40-51c17c4dba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:11.856605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c5192-1db9-431c-bb40-51c17c4dba6a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:11.856884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c5192-1db9-431c-bb40-51c17c4dba6a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:11.857151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c5192-1db9-431c-bb40-51c17c4dba6a None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:11.857840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c5192-1db9-431c-bb40-51c17c4dba6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:11.858143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c5192-1db9-431c-bb40-51c17c4dba6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:11.859486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c5192-1db9-431c-bb40-51c17c4dba6a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:11.863655 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 278/567] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:11.864483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707c2a3-12c3-47f4-85a2-d08c3d9303ce None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:11.875778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 290/568] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:11.875778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c5192-1db9-431c-bb40-51c17c4dba6a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:11.933747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:11.976744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707c2a3-12c3-47f4-85a2-d08c3d9303ce None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:11.977243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c5192-1db9-431c-bb40-51c17c4dba6a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.011618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e208-ee03-4c45-9635-923979ac00ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.012326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e208-ee03-4c45-9635-923979ac00ee None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.012526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e208-ee03-4c45-9635-923979ac00ee None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.012783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e208-ee03-4c45-9635-923979ac00ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.013745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9997a5-0f03-44ba-805e-ef06789524ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.014366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9997a5-0f03-44ba-805e-ef06789524ea None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.014644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9997a5-0f03-44ba-805e-ef06789524ea None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.014913 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9997a5-0f03-44ba-805e-ef06789524ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.021186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0e208-ee03-4c45-9635-923979ac00ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.022646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0e208-ee03-4c45-9635-923979ac00ee None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.026074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9997a5-0f03-44ba-805e-ef06789524ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-9a9997a5-0f03-44ba-805e-ef06789524ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:12.030840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:12.031746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:12.033721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 279/569] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:12.034440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9997a5-0f03-44ba-805e-ef06789524ea None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-53a0e208-ee03-4c45-9635-923979ac00ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:12.036977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:12.038128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:12.039117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:12.039117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:12.039117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:12.039117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:12.039117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:12.039117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:12.039117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:12.039117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:12.039117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:12.039117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:12.044904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 291/570] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 1 05:13:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:12.044904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e208-ee03-4c45-9635-923979ac00ee None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.050134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:12.145998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e208-ee03-4c45-9635-923979ac00ee None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.149232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9997a5-0f03-44ba-805e-ef06789524ea None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.163251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:12.180315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e5e1f-d952-4f8f-a7c3-6fc9269939f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.180946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5e1f-d952-4f8f-a7c3-6fc9269939f4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.181225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5e1f-d952-4f8f-a7c3-6fc9269939f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.181527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5e1f-d952-4f8f-a7c3-6fc9269939f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.183052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22deca87-2211-4e7c-9b95-0b14bdc9bb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.183672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deca87-2211-4e7c-9b95-0b14bdc9bb3b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.183953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deca87-2211-4e7c-9b95-0b14bdc9bb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.184226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deca87-2211-4e7c-9b95-0b14bdc9bb3b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.188059 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e5e1f-d952-4f8f-a7c3-6fc9269939f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.189810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e5e1f-d952-4f8f-a7c3-6fc9269939f4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.191303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22deca87-2211-4e7c-9b95-0b14bdc9bb3b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.192742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22deca87-2211-4e7c-9b95-0b14bdc9bb3b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-581e5e1f-d952-4f8f-a7c3-6fc9269939f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:12.201927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:12.202980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:12.204063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:12.204063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:12.204063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:12.204063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:12.204063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:12.204063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:12.204063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:12.204063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:12.204063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 292/571] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:12.204063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e5e1f-d952-4f8f-a7c3-6fc9269939f4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-22deca87-2211-4e7c-9b95-0b14bdc9bb3b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:12.207077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:12.208274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 280/572] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:12.210068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22deca87-2211-4e7c-9b95-0b14bdc9bb3b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.282555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:12.309394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e5e1f-d952-4f8f-a7c3-6fc9269939f4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.311951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22deca87-2211-4e7c-9b95-0b14bdc9bb3b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.343178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc9444-a27c-4975-a056-6d03ccc5e99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.343786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc9444-a27c-4975-a056-6d03ccc5e99b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.344065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc9444-a27c-4975-a056-6d03ccc5e99b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.344336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc9444-a27c-4975-a056-6d03ccc5e99b None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.345071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc9444-a27c-4975-a056-6d03ccc5e99b None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:12.345301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc9444-a27c-4975-a056-6d03ccc5e99b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.346106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb50cba-7c1f-4f39-936d-b91fd0799010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.346704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb50cba-7c1f-4f39-936d-b91fd0799010 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.346849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc9444-a27c-4975-a056-6d03ccc5e99b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.346982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb50cba-7c1f-4f39-936d-b91fd0799010 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.347254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb50cba-7c1f-4f39-936d-b91fd0799010 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.347980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb50cba-7c1f-4f39-936d-b91fd0799010 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:12.348455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb50cba-7c1f-4f39-936d-b91fd0799010 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.349897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb50cba-7c1f-4f39-936d-b91fd0799010 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.354715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 293/573] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:12.355511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc9444-a27c-4975-a056-6d03ccc5e99b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.374686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 281/574] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:12.374686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb50cba-7c1f-4f39-936d-b91fd0799010 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.396736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:12.455608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc9444-a27c-4975-a056-6d03ccc5e99b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.471565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb50cba-7c1f-4f39-936d-b91fd0799010 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.489385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d68ad2-3f00-4a3a-a54b-9e3194d7bc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.490039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d68ad2-3f00-4a3a-a54b-9e3194d7bc28 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.490319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d68ad2-3f00-4a3a-a54b-9e3194d7bc28 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.490591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d68ad2-3f00-4a3a-a54b-9e3194d7bc28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.500454 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d68ad2-3f00-4a3a-a54b-9e3194d7bc28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:12.500984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e7d68ad2-3f00-4a3a-a54b-9e3194d7bc28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:12.505750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:12.506757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:12.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:12.508415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f99fd-6794-415c-957d-0d42632e78e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.508505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 294/575] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:12.508601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f99fd-6794-415c-957d-0d42632e78e5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.508675 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d68ad2-3f00-4a3a-a54b-9e3194d7bc28 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.509132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f99fd-6794-415c-957d-0d42632e78e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.509132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f99fd-6794-415c-957d-0d42632e78e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.509132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f99fd-6794-415c-957d-0d42632e78e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:12.509132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f99fd-6794-415c-957d-0d42632e78e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.510383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f99fd-6794-415c-957d-0d42632e78e5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.523384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 282/576] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:12.524187 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f99fd-6794-415c-957d-0d42632e78e5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.605443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:12.608292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d68ad2-3f00-4a3a-a54b-9e3194d7bc28 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.625389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f99fd-6794-415c-957d-0d42632e78e5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.643398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84d4bc-6b76-4c15-b77e-129214365b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.644007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84d4bc-6b76-4c15-b77e-129214365b45 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.644287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84d4bc-6b76-4c15-b77e-129214365b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.644561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84d4bc-6b76-4c15-b77e-129214365b45 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.645285 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84d4bc-6b76-4c15-b77e-129214365b45 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:12.645625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84d4bc-6b76-4c15-b77e-129214365b45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.647039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84d4bc-6b76-4c15-b77e-129214365b45 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.661266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8937fac-621c-4032-a723-470db62066ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.661907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8937fac-621c-4032-a723-470db62066ee None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.662179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8937fac-621c-4032-a723-470db62066ee None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.662443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8937fac-621c-4032-a723-470db62066ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.664486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 295/577] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:12.665278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84d4bc-6b76-4c15-b77e-129214365b45 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.678623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8937fac-621c-4032-a723-470db62066ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.680093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8937fac-621c-4032-a723-470db62066ee None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d8937fac-621c-4032-a723-470db62066ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:12.694253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:12.695277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:12.696186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:12.696186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:12.696186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:12.696186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 283/578] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:12.696186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8937fac-621c-4032-a723-470db62066ee None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.743376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:12.769309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84d4bc-6b76-4c15-b77e-129214365b45 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.800016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8937fac-621c-4032-a723-470db62066ee None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.803755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9da6c1-5f5d-423b-aaba-9ad5d35bf163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.804391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9da6c1-5f5d-423b-aaba-9ad5d35bf163 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.804672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9da6c1-5f5d-423b-aaba-9ad5d35bf163 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.804942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9da6c1-5f5d-423b-aaba-9ad5d35bf163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.816612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9da6c1-5f5d-423b-aaba-9ad5d35bf163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.818078 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9da6c1-5f5d-423b-aaba-9ad5d35bf163 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1b9da6c1-5f5d-423b-aaba-9ad5d35bf163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:12.832635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:12.833790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:12.834876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:12.834876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:12.834876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:12.834876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 296/579] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:12.835177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd977d7c-67c4-46ae-a6d7-cddada05e4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.835268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9da6c1-5f5d-423b-aaba-9ad5d35bf163 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.835464 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd977d7c-67c4-46ae-a6d7-cddada05e4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.835742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd977d7c-67c4-46ae-a6d7-cddada05e4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.836014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd977d7c-67c4-46ae-a6d7-cddada05e4a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.836676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd977d7c-67c4-46ae-a6d7-cddada05e4a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:12.836985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd977d7c-67c4-46ae-a6d7-cddada05e4a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.838418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd977d7c-67c4-46ae-a6d7-cddada05e4a7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.846678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 284/580] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:12.847482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd977d7c-67c4-46ae-a6d7-cddada05e4a7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.850282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:12.937486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9da6c1-5f5d-423b-aaba-9ad5d35bf163 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.948089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd977d7c-67c4-46ae-a6d7-cddada05e4a7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:12.950641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:12.971355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18862b24-b3c3-44a2-aaa3-0b3fa2a5ca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.971950 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18862b24-b3c3-44a2-aaa3-0b3fa2a5ca9e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.972229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18862b24-b3c3-44a2-aaa3-0b3fa2a5ca9e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.972494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18862b24-b3c3-44a2-aaa3-0b3fa2a5ca9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:12.973203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18862b24-b3c3-44a2-aaa3-0b3fa2a5ca9e None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=b723681faf15434e8529e8fc56c6e9b8) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:12.973532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18862b24-b3c3-44a2-aaa3-0b3fa2a5ca9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:12.974957 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18862b24-b3c3-44a2-aaa3-0b3fa2a5ca9e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:12.980310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 297/581] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/role_assignments?user.id=b723681faf15434e8529e8fc56c6e9b8&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:12.980909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18862b24-b3c3-44a2-aaa3-0b3fa2a5ca9e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:12.995416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43718b6-cf98-4e77-832f-44211eab9b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:12.996074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43718b6-cf98-4e77-832f-44211eab9b3c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:12.996354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43718b6-cf98-4e77-832f-44211eab9b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:12.996629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43718b6-cf98-4e77-832f-44211eab9b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.008471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43718b6-cf98-4e77-832f-44211eab9b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.010130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43718b6-cf98-4e77-832f-44211eab9b3c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d43718b6-cf98-4e77-832f-44211eab9b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:13.027090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:13.028344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:13.029737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:13.029737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:13.029737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:13.029737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 285/582] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:13.029737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43718b6-cf98-4e77-832f-44211eab9b3c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:13.072239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:13.100182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18862b24-b3c3-44a2-aaa3-0b3fa2a5ca9e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:13.140747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53f088-f56c-42e8-a071-d2ea4071612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:13.141554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53f088-f56c-42e8-a071-d2ea4071612e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:13.142472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53f088-f56c-42e8-a071-d2ea4071612e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:13.143086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53f088-f56c-42e8-a071-d2ea4071612e None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.143737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53f088-f56c-42e8-a071-d2ea4071612e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:13.144043 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53f088-f56c-42e8-a071-d2ea4071612e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.145421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53f088-f56c-42e8-a071-d2ea4071612e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.149181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43718b6-cf98-4e77-832f-44211eab9b3c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:13.157380 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 298/583] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:13.158230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53f088-f56c-42e8-a071-d2ea4071612e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:13.192040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706c3eb-4da1-4723-9b4b-1e9ab0f9eee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:13.192571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706c3eb-4da1-4723-9b4b-1e9ab0f9eee0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:13.192867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706c3eb-4da1-4723-9b4b-1e9ab0f9eee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:13.193139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706c3eb-4da1-4723-9b4b-1e9ab0f9eee0 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.193820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706c3eb-4da1-4723-9b4b-1e9ab0f9eee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:13.194124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706c3eb-4da1-4723-9b4b-1e9ab0f9eee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.195503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706c3eb-4da1-4723-9b4b-1e9ab0f9eee0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.208744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 286/584] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:13.209406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706c3eb-4da1-4723-9b4b-1e9ab0f9eee0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:13.245164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:13.268934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53f088-f56c-42e8-a071-d2ea4071612e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:13.315684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74e619-80ff-45fe-a20e-ba1263667499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:13.316419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74e619-80ff-45fe-a20e-ba1263667499 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:13.316760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74e619-80ff-45fe-a20e-ba1263667499 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:13.317092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74e619-80ff-45fe-a20e-ba1263667499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.319321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:13.328709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74e619-80ff-45fe-a20e-ba1263667499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.330238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74e619-80ff-45fe-a20e-ba1263667499 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.335127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706c3eb-4da1-4723-9b4b-1e9ab0f9eee0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ed74e619-80ff-45fe-a20e-ba1263667499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:13.345592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:13.346744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:13.347864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:13.347864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:13.347864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:13.351471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 299/585] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:13.352172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74e619-80ff-45fe-a20e-ba1263667499 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:13.374785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78124c23-a03c-438a-a9af-a7f16a67c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:13.375523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78124c23-a03c-438a-a9af-a7f16a67c7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:13.375883 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78124c23-a03c-438a-a9af-a7f16a67c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:13.376350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78124c23-a03c-438a-a9af-a7f16a67c7a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.377179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78124c23-a03c-438a-a9af-a7f16a67c7a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:13.377623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78124c23-a03c-438a-a9af-a7f16a67c7a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.379086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78124c23-a03c-438a-a9af-a7f16a67c7a9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.391305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 287/586] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:13.392980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78124c23-a03c-438a-a9af-a7f16a67c7a9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:13.470720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74e619-80ff-45fe-a20e-ba1263667499 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:13.473886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:13.498975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78124c23-a03c-438a-a9af-a7f16a67c7a9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:13.508828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f00f0-d9a6-4158-9c82-36f6026ac25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:13.509663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f00f0-d9a6-4158-9c82-36f6026ac25f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:13.509955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f00f0-d9a6-4158-9c82-36f6026ac25f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:13.510326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f00f0-d9a6-4158-9c82-36f6026ac25f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.519462 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f00f0-d9a6-4158-9c82-36f6026ac25f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.521218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f00f0-d9a6-4158-9c82-36f6026ac25f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4b4f00f0-d9a6-4158-9c82-36f6026ac25f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:13.533134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:13.534269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:13.535426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:13.535426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:13.535426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:13.535426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:13.535426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:13.535426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:13.535426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:13.535426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:13.535426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:13:13.535426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:13.540569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 300/587] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:13.540569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f00f0-d9a6-4158-9c82-36f6026ac25f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:13.542534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381ddb9-41c5-46f4-999e-202948dfae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:13.543321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ddb9-41c5-46f4-999e-202948dfae6c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:13.543704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ddb9-41c5-46f4-999e-202948dfae6c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:13.544073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ddb9-41c5-46f4-999e-202948dfae6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.552166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381ddb9-41c5-46f4-999e-202948dfae6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.553757 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381ddb9-41c5-46f4-999e-202948dfae6c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-4381ddb9-41c5-46f4-999e-202948dfae6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:13.565307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:13.566410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:13.567531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:13.567531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:13.567531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:13.567531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:13.567531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:13.567531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:13.567531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:13.567531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:13.572816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 288/588] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:13.572816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381ddb9-41c5-46f4-999e-202948dfae6c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:13.596793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:13.663229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f00f0-d9a6-4158-9c82-36f6026ac25f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:13.688891 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381ddb9-41c5-46f4-999e-202948dfae6c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:13.699884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2806c58-8a87-41e9-9219-887d2ee1e144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:13.700592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2806c58-8a87-41e9-9219-887d2ee1e144 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:13.700944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2806c58-8a87-41e9-9219-887d2ee1e144 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:13.701287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2806c58-8a87-41e9-9219-887d2ee1e144 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.702152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2806c58-8a87-41e9-9219-887d2ee1e144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:13.702651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2806c58-8a87-41e9-9219-887d2ee1e144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.704112 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2806c58-8a87-41e9-9219-887d2ee1e144 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.725399 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 301/589] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:13.725399 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2806c58-8a87-41e9-9219-887d2ee1e144 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:13.735938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e87c9-de9f-4159-a7d4-ae91f6a57eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:13.736822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e87c9-de9f-4159-a7d4-ae91f6a57eac None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:13.737480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e87c9-de9f-4159-a7d4-ae91f6a57eac None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:13.737480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e87c9-de9f-4159-a7d4-ae91f6a57eac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.740439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:13.745626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e87c9-de9f-4159-a7d4-ae91f6a57eac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.747072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e87c9-de9f-4159-a7d4-ae91f6a57eac None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cd7e87c9-de9f-4159-a7d4-ae91f6a57eac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:13.758943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:13.759855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:13.760795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:13.760795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:13.760795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:13.760795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:13.760795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:13.760795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:13.760795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:13.760795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:13.760795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 289/590] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:13.760795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e87c9-de9f-4159-a7d4-ae91f6a57eac None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:13.839525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2806c58-8a87-41e9-9219-887d2ee1e144 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:13.847890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:13.871336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e87c9-de9f-4159-a7d4-ae91f6a57eac None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:13.875482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6693cc-a7eb-486c-93cf-e85f520f70d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:13.876695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6693cc-a7eb-486c-93cf-e85f520f70d3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:13.877198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6693cc-a7eb-486c-93cf-e85f520f70d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:13.877571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6693cc-a7eb-486c-93cf-e85f520f70d3 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.878372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6693cc-a7eb-486c-93cf-e85f520f70d3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:13.878752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6693cc-a7eb-486c-93cf-e85f520f70d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.880246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6693cc-a7eb-486c-93cf-e85f520f70d3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.899941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 302/591] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:13.900943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6693cc-a7eb-486c-93cf-e85f520f70d3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:13.914898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8eaef-5160-434b-ab8b-edaf19844f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:13.915606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8eaef-5160-434b-ab8b-edaf19844f57 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:13.915888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8eaef-5160-434b-ab8b-edaf19844f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:13.916169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8eaef-5160-434b-ab8b-edaf19844f57 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:13.916864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8eaef-5160-434b-ab8b-edaf19844f57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:13.917167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8eaef-5160-434b-ab8b-edaf19844f57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:13.918624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8eaef-5160-434b-ab8b-edaf19844f57 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:13.932508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 290/592] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:13.933351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8eaef-5160-434b-ab8b-edaf19844f57 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:14.000087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:14.012007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6693cc-a7eb-486c-93cf-e85f520f70d3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:14.046495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8eaef-5160-434b-ab8b-edaf19844f57 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:14.050920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc3d67-df69-4465-b7b6-8fa42321f505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:14.051558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc3d67-df69-4465-b7b6-8fa42321f505 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:14.051842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc3d67-df69-4465-b7b6-8fa42321f505 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:14.052119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc3d67-df69-4465-b7b6-8fa42321f505 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:14.059778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc3d67-df69-4465-b7b6-8fa42321f505 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:14.061261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc3d67-df69-4465-b7b6-8fa42321f505 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d5bc3d67-df69-4465-b7b6-8fa42321f505 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:14.075203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:14.076852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:14.078049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:14.078049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:14.078049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:14.078049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:14.078049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:14.078049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:14.078049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:14.078049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:14.078049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:13:14.078049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:14.081925 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 303/593] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:14.083353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc3d67-df69-4465-b7b6-8fa42321f505 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:14.092833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1648677-01d3-43e5-a362-a01ebdc75d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:14.093439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648677-01d3-43e5-a362-a01ebdc75d32 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:14.093750 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648677-01d3-43e5-a362-a01ebdc75d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:14.094020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648677-01d3-43e5-a362-a01ebdc75d32 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:14.094709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1648677-01d3-43e5-a362-a01ebdc75d32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:14.095064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1648677-01d3-43e5-a362-a01ebdc75d32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:14.096496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1648677-01d3-43e5-a362-a01ebdc75d32 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:14.113786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 291/594] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:14.113786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1648677-01d3-43e5-a362-a01ebdc75d32 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:14.116348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:14.203685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc3d67-df69-4465-b7b6-8fa42321f505 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:14.223382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1648677-01d3-43e5-a362-a01ebdc75d32 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:14.241960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:14.243714 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b637235-44b7-49f9-9ec7-4265eec4dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:14.244348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b637235-44b7-49f9-9ec7-4265eec4dd71 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:14.244623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b637235-44b7-49f9-9ec7-4265eec4dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:14.244888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b637235-44b7-49f9-9ec7-4265eec4dd71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:14.255831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b637235-44b7-49f9-9ec7-4265eec4dd71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:14.257199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b637235-44b7-49f9-9ec7-4265eec4dd71 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:14.261175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8dffa-1c9e-4ca3-a2de-88c398de7141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:14.261935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8dffa-1c9e-4ca3-a2de-88c398de7141 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:14.262282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8dffa-1c9e-4ca3-a2de-88c398de7141 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:14.262640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8dffa-1c9e-4ca3-a2de-88c398de7141 None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/7cda3c1f67e54ee08abc014399b07cc5/roles/c6689cf082df4be9b69f2208c097f2c3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0b637235-44b7-49f9-9ec7-4265eec4dd71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:14.271774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:14.272928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:14.274471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:14.274471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:14.274471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:14.277677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 304/595] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:14.278352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b637235-44b7-49f9-9ec7-4265eec4dd71 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:14.284140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8dffa-1c9e-4ca3-a2de-88c398de7141 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=7cda3c1f67e54ee08abc014399b07cc5, role_id=c6689cf082df4be9b69f2208c097f2c3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:14.285694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8dffa-1c9e-4ca3-a2de-88c398de7141 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:14.355291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:14.363961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 292/596] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:13:14 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/7cda3c1f67e54ee08abc014399b07cc5/roles/c6689cf082df4be9b69f2208c097f2c3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:13:14.365020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8dffa-1c9e-4ca3-a2de-88c398de7141 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:14.389992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b637235-44b7-49f9-9ec7-4265eec4dd71 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:14.434880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e044d-84ab-4b5d-b500-5c91243cbc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:14.435520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e044d-84ab-4b5d-b500-5c91243cbc00 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:14.435799 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e044d-84ab-4b5d-b500-5c91243cbc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:14.436070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e044d-84ab-4b5d-b500-5c91243cbc00 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:14.436832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e044d-84ab-4b5d-b500-5c91243cbc00 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:14.437143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e044d-84ab-4b5d-b500-5c91243cbc00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:14.439690 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e044d-84ab-4b5d-b500-5c91243cbc00 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:14.460322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 305/597] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Sep 1 05:13:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:14.461329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e044d-84ab-4b5d-b500-5c91243cbc00 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:14.498737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8dffa-1c9e-4ca3-a2de-88c398de7141 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:14.540709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b0c1-3bfa-4218-a314-c6593f5e344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:14.541349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b0c1-3bfa-4218-a314-c6593f5e344a None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:14.541680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b0c1-3bfa-4218-a314-c6593f5e344a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:14.541996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b0c1-3bfa-4218-a314-c6593f5e344a None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/4c23597e0da2454abd2a5354d779b092/roles/c6689cf082df4be9b69f2208c097f2c3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:14.559534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682b0c1-3bfa-4218-a314-c6593f5e344a None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=4c23597e0da2454abd2a5354d779b092, role_id=c6689cf082df4be9b69f2208c097f2c3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:14.561053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682b0c1-3bfa-4218-a314-c6593f5e344a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:14.592071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e044d-84ab-4b5d-b500-5c91243cbc00 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:14.635568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 293/598] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:13:14 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/4c23597e0da2454abd2a5354d779b092/roles/c6689cf082df4be9b69f2208c097f2c3 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:13:14.641499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b0c1-3bfa-4218-a314-c6593f5e344a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:14.644245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd4825-c713-477b-b5ef-48a5b3fb312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:14.644890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd4825-c713-477b-b5ef-48a5b3fb312d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:14.645213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd4825-c713-477b-b5ef-48a5b3fb312d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:14.645532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd4825-c713-477b-b5ef-48a5b3fb312d None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:14.646259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd4825-c713-477b-b5ef-48a5b3fb312d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:14.646623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd4825-c713-477b-b5ef-48a5b3fb312d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:14.648627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd4825-c713-477b-b5ef-48a5b3fb312d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:14.654752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:14.678020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 306/599] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:14.683838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd4825-c713-477b-b5ef-48a5b3fb312d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:14.693823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:14.843973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd4825-c713-477b-b5ef-48a5b3fb312d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:14.867477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b0c1-3bfa-4218-a314-c6593f5e344a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:14.882578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb836f-e084-4957-8f28-a3c7bc26b681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:14.883214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb836f-e084-4957-8f28-a3c7bc26b681 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:14.883489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb836f-e084-4957-8f28-a3c7bc26b681 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:14.883859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb836f-e084-4957-8f28-a3c7bc26b681 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:14.891107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb836f-e084-4957-8f28-a3c7bc26b681 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:14.893693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb836f-e084-4957-8f28-a3c7bc26b681 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f5fb836f-e084-4957-8f28-a3c7bc26b681 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:14.905860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:14.906815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:14.907785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:14.907785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:14.907785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:14.907785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:14.907785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:14.907785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:14.907785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:14.907785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:14.907785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 307/600] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:14.913051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb836f-e084-4957-8f28-a3c7bc26b681 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:14.917662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4e6db-5e89-4135-99b5-ca27aa6cde2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:14.918310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4e6db-5e89-4135-99b5-ca27aa6cde2d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:14.918590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4e6db-5e89-4135-99b5-ca27aa6cde2d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:14.918864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4e6db-5e89-4135-99b5-ca27aa6cde2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:14.931590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4e6db-5e89-4135-99b5-ca27aa6cde2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:14.933208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4e6db-5e89-4135-99b5-ca27aa6cde2d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d2a4e6db-5e89-4135-99b5-ca27aa6cde2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:14.967658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:14.970010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:14.972380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:14.972380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:14.972380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:14.977479 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 294/601] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:14.979320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4e6db-5e89-4135-99b5-ca27aa6cde2d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:14.999651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:15.045738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb836f-e084-4957-8f28-a3c7bc26b681 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:15.076790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:15.094907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecb390-dbc8-4389-a83c-1942c70608d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:15.096239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb390-dbc8-4389-a83c-1942c70608d2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:15.096685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb390-dbc8-4389-a83c-1942c70608d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:15.097370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb390-dbc8-4389-a83c-1942c70608d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:15.098256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecb390-dbc8-4389-a83c-1942c70608d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:15.098779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecb390-dbc8-4389-a83c-1942c70608d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:15.100902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecb390-dbc8-4389-a83c-1942c70608d2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:15.113069 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4e6db-5e89-4135-99b5-ca27aa6cde2d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:15.121068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 308/602] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:15.122645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cda76e-6381-4a81-8a9e-40559ef402c3 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:15.123106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cda76e-6381-4a81-8a9e-40559ef402c3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:15.123522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cda76e-6381-4a81-8a9e-40559ef402c3 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:15.125145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 309/603] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:15.128526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cda76e-6381-4a81-8a9e-40559ef402c3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:15.159690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32711e7-cca6-4f94-ae6e-b433f3018744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:15.160445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32711e7-cca6-4f94-ae6e-b433f3018744 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:15.160822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32711e7-cca6-4f94-ae6e-b433f3018744 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:15.161189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32711e7-cca6-4f94-ae6e-b433f3018744 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:15.161967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32711e7-cca6-4f94-ae6e-b433f3018744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:15.162366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32711e7-cca6-4f94-ae6e-b433f3018744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:15.163839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32711e7-cca6-4f94-ae6e-b433f3018744 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:15.178114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 295/604] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:15.179432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210a2bc-b244-4f15-a080-420cffbc00f4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:15.179826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210a2bc-b244-4f15-a080-420cffbc00f4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:15.180267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210a2bc-b244-4f15-a080-420cffbc00f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:15.199909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6210a2bc-b244-4f15-a080-420cffbc00f4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:15.201178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6210a2bc-b244-4f15-a080-420cffbc00f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:15.261364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cda76e-6381-4a81-8a9e-40559ef402c3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:15.280510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:15.296133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 296/605] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:15.297609 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210a2bc-b244-4f15-a080-420cffbc00f4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:15.304512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f4838-9ee1-4b2b-8529-27130e83f07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:15.305276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f4838-9ee1-4b2b-8529-27130e83f07c None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:15.305646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f4838-9ee1-4b2b-8529-27130e83f07c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:15.305922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f4838-9ee1-4b2b-8529-27130e83f07c None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/7e0e8589931b46ca84168a2ad273b63d/roles/c6689cf082df4be9b69f2208c097f2c3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:15.334828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f4838-9ee1-4b2b-8529-27130e83f07c None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=7e0e8589931b46ca84168a2ad273b63d, role_id=c6689cf082df4be9b69f2208c097f2c3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:15.337433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f4838-9ee1-4b2b-8529-27130e83f07c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:15.380159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:15.426236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210a2bc-b244-4f15-a080-420cffbc00f4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:15.443983 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 310/606] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:13:15 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/7e0e8589931b46ca84168a2ad273b63d/roles/c6689cf082df4be9b69f2208c097f2c3 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:13:15.493313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4659f-a000-4d4d-8a47-9a6b25f13c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:15.494388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4659f-a000-4d4d-8a47-9a6b25f13c8e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:15.494875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4659f-a000-4d4d-8a47-9a6b25f13c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:15.495384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4659f-a000-4d4d-8a47-9a6b25f13c8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:15.505782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4659f-a000-4d4d-8a47-9a6b25f13c8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:15.507723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4659f-a000-4d4d-8a47-9a6b25f13c8e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-34a4659f-a000-4d4d-8a47-9a6b25f13c8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:15.521535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:15.522575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:15.523467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:15.523467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:15.523467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:15.523467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:15.523467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:15.523467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:15.523467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:15.523467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:15.526133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 297/607] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:15.532777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f4838-9ee1-4b2b-8529-27130e83f07c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:15.599479 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4659f-a000-4d4d-8a47-9a6b25f13c8e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:15.685379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f4838-9ee1-4b2b-8529-27130e83f07c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:15.736727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664dba1-eefc-4c77-922b-8a14520b168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:15.737633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664dba1-eefc-4c77-922b-8a14520b168c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:15.737917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664dba1-eefc-4c77-922b-8a14520b168c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:15.738615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664dba1-eefc-4c77-922b-8a14520b168c None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:15.739590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664dba1-eefc-4c77-922b-8a14520b168c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:15.739899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664dba1-eefc-4c77-922b-8a14520b168c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:15.742256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664dba1-eefc-4c77-922b-8a14520b168c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:15.744319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4659f-a000-4d4d-8a47-9a6b25f13c8e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:15.757535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 311/608] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:15.765278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:15.766509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664dba1-eefc-4c77-922b-8a14520b168c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:15.785572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873bb92-b732-4ec4-a942-92e72e16155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:15.786250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873bb92-b732-4ec4-a942-92e72e16155f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:15.786583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873bb92-b732-4ec4-a942-92e72e16155f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:15.786840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873bb92-b732-4ec4-a942-92e72e16155f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:15.794744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:15.799385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873bb92-b732-4ec4-a942-92e72e16155f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-5873bb92-b732-4ec4-a942-92e72e16155f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:15.804303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:15.805312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:15.807208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 298/609] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:15.811955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873bb92-b732-4ec4-a942-92e72e16155f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:15.872891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664dba1-eefc-4c77-922b-8a14520b168c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:15.920622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374679b-771b-4150-8a28-ade0207d7be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:15.921294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374679b-771b-4150-8a28-ade0207d7be1 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:15.921595 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374679b-771b-4150-8a28-ade0207d7be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:15.921868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374679b-771b-4150-8a28-ade0207d7be1 None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/0e21910fca40479ca04e7f0418f7dd2d/roles/c6689cf082df4be9b69f2208c097f2c3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:15.947084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374679b-771b-4150-8a28-ade0207d7be1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=0e21910fca40479ca04e7f0418f7dd2d, role_id=c6689cf082df4be9b69f2208c097f2c3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:15.948443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374679b-771b-4150-8a28-ade0207d7be1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:15.957032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873bb92-b732-4ec4-a942-92e72e16155f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:15.994132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1bb8-bc3f-4551-b16f-b139dfffd932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:15.994743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1bb8-bc3f-4551-b16f-b139dfffd932 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:15.995021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1bb8-bc3f-4551-b16f-b139dfffd932 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:15.995321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1bb8-bc3f-4551-b16f-b139dfffd932 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:15.995977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1bb8-bc3f-4551-b16f-b139dfffd932 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:15.996279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1bb8-bc3f-4551-b16f-b139dfffd932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:15.997676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1bb8-bc3f-4551-b16f-b139dfffd932 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:16.007789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 299/610] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:16.013658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1bb8-bc3f-4551-b16f-b139dfffd932 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:16.030026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 312/611] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:13:15 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/0e21910fca40479ca04e7f0418f7dd2d/roles/c6689cf082df4be9b69f2208c097f2c3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:13:16.063119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:16.128720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1bb8-bc3f-4551-b16f-b139dfffd932 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:16.167797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0d04e-b48c-499a-9191-d4b6c48bc8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:16.168443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0d04e-b48c-499a-9191-d4b6c48bc8fb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:16.168727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0d04e-b48c-499a-9191-d4b6c48bc8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:16.169066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0d04e-b48c-499a-9191-d4b6c48bc8fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:16.181694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0d04e-b48c-499a-9191-d4b6c48bc8fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:16.183196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0d04e-b48c-499a-9191-d4b6c48bc8fb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-2eb0d04e-b48c-499a-9191-d4b6c48bc8fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:16.200441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:16.201429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:16.203634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:16.203634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:16.203634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:16.203634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 300/612] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:16.210613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374679b-771b-4150-8a28-ade0207d7be1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:16.303499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:16.331160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374679b-771b-4150-8a28-ade0207d7be1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:16.370656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73949e37-15b9-4bb7-9dad-26b73b563911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:16.371264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73949e37-15b9-4bb7-9dad-26b73b563911 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:16.371606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73949e37-15b9-4bb7-9dad-26b73b563911 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:16.371880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73949e37-15b9-4bb7-9dad-26b73b563911 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:16.372529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73949e37-15b9-4bb7-9dad-26b73b563911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:16.372833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73949e37-15b9-4bb7-9dad-26b73b563911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:16.374231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73949e37-15b9-4bb7-9dad-26b73b563911 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:16.388860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 313/613] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:16.395129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0d04e-b48c-499a-9191-d4b6c48bc8fb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:16.493948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:16.503525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0d04e-b48c-499a-9191-d4b6c48bc8fb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:16.538752 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacb25d-3ce2-44e9-a2a1-61d949dd0a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:16.539455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacb25d-3ce2-44e9-a2a1-61d949dd0a48 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:16.539997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacb25d-3ce2-44e9-a2a1-61d949dd0a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:16.540279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacb25d-3ce2-44e9-a2a1-61d949dd0a48 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:16.548460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bacb25d-3ce2-44e9-a2a1-61d949dd0a48 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:16.550185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bacb25d-3ce2-44e9-a2a1-61d949dd0a48 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-0bacb25d-3ce2-44e9-a2a1-61d949dd0a48 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:16.562759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:16.565690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:16.568011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 301/614] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:16.574251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73949e37-15b9-4bb7-9dad-26b73b563911 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:16.700280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73949e37-15b9-4bb7-9dad-26b73b563911 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:16.750292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ea1ea-2bcf-44c8-bf39-0d83f7016ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:16.750834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ea1ea-2bcf-44c8-bf39-0d83f7016ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:16.751726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ea1ea-2bcf-44c8-bf39-0d83f7016ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:16.752000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ea1ea-2bcf-44c8-bf39-0d83f7016ea3 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:16.752744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ea1ea-2bcf-44c8-bf39-0d83f7016ea3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:16.753047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ea1ea-2bcf-44c8-bf39-0d83f7016ea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:16.754505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ea1ea-2bcf-44c8-bf39-0d83f7016ea3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:16.756361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:16.783148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 314/615] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 1 05:13:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:16.788634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacb25d-3ce2-44e9-a2a1-61d949dd0a48 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:16.898564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacb25d-3ce2-44e9-a2a1-61d949dd0a48 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:16.934480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd1dde-ee52-4938-9c29-e806e53d47bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:16.935109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd1dde-ee52-4938-9c29-e806e53d47bf None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:16.935390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd1dde-ee52-4938-9c29-e806e53d47bf None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:16.935665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd1dde-ee52-4938-9c29-e806e53d47bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:16.947408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd1dde-ee52-4938-9c29-e806e53d47bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:16.949154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd1dde-ee52-4938-9c29-e806e53d47bf None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-44dd1dde-ee52-4938-9c29-e806e53d47bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:16.964839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:16.967204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:16.969331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:16.969331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:16.969331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:16.969331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 302/616] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:16.975362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ea1ea-2bcf-44c8-bf39-0d83f7016ea3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:17.093397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ea1ea-2bcf-44c8-bf39-0d83f7016ea3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:17.128281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ab967-78e3-4b62-a5ed-b4b79a527bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:17.128903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ab967-78e3-4b62-a5ed-b4b79a527bca None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.129185 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ab967-78e3-4b62-a5ed-b4b79a527bca None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.129484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ab967-78e3-4b62-a5ed-b4b79a527bca None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:17.130129 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ab967-78e3-4b62-a5ed-b4b79a527bca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:17.130431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ab967-78e3-4b62-a5ed-b4b79a527bca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:17.132153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ab967-78e3-4b62-a5ed-b4b79a527bca None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:17.133310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25f10-496b-4856-890c-20b9b81d0bcc None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.133781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25f10-496b-4856-890c-20b9b81d0bcc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.133915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25f10-496b-4856-890c-20b9b81d0bcc None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:17.135384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 303/617] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:17.143850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3781c55-cc3f-4bcf-b6bf-becd9eaf1a1e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.144133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3781c55-cc3f-4bcf-b6bf-becd9eaf1a1e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.144402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3781c55-cc3f-4bcf-b6bf-becd9eaf1a1e None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:17.145968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 304/618] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:17.146979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1785603-2c29-41a6-8455-15eaac6bede8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.147258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1785603-2c29-41a6-8455-15eaac6bede8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.147530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1785603-2c29-41a6-8455-15eaac6bede8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:17.151230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 315/619] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:17.156036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba1af-7531-4ba4-810d-4f6f32900749 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.156429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba1af-7531-4ba4-810d-4f6f32900749 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.156707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba1af-7531-4ba4-810d-4f6f32900749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:17.170251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c1785603-2c29-41a6-8455-15eaac6bede8 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:17.171468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c1785603-2c29-41a6-8455-15eaac6bede8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:17.174402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:17.175918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-541ba1af-7531-4ba4-810d-4f6f32900749 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:17.177156 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-541ba1af-7531-4ba4-810d-4f6f32900749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:17.227474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:17.253679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 316/621] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:13:17.253679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ba1af-7531-4ba4-810d-4f6f32900749 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:17.254160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 305/621] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:17.353979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ba1af-7531-4ba4-810d-4f6f32900749 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:17.389909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f76975-0027-4782-828a-f2e4c3d60ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:17.390547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f76975-0027-4782-828a-f2e4c3d60ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.390832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f76975-0027-4782-828a-f2e4c3d60ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.391103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f76975-0027-4782-828a-f2e4c3d60ee0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:17.398876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f76975-0027-4782-828a-f2e4c3d60ee0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:17.400327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f76975-0027-4782-828a-f2e4c3d60ee0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c3f76975-0027-4782-828a-f2e4c3d60ee0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:17.412166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:17.413138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:17.414113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:17.414113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:17.414113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:17.414113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:17.414113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:17.414113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:17.414113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:17.414113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:17.414113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 317/622] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:17.419038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1785603-2c29-41a6-8455-15eaac6bede8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:17.586617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1785603-2c29-41a6-8455-15eaac6bede8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:17.618785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f76975-0027-4782-828a-f2e4c3d60ee0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:17.645981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f3b9e-0d37-4794-a96f-a282c5874522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:17.646843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f3b9e-0d37-4794-a96f-a282c5874522 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.647268 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f3b9e-0d37-4794-a96f-a282c5874522 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.647696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f3b9e-0d37-4794-a96f-a282c5874522 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:17.648670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f3b9e-0d37-4794-a96f-a282c5874522 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:17.649183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f3b9e-0d37-4794-a96f-a282c5874522 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:17.650921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f3b9e-0d37-4794-a96f-a282c5874522 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:17.652919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:17.669066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:17.722480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 306/623] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:17.722480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f3b9e-0d37-4794-a96f-a282c5874522 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:17.772747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f76975-0027-4782-828a-f2e4c3d60ee0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:17.812931 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0e837-294c-4f39-94bd-eef22fa7f87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:17.813814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0e837-294c-4f39-94bd-eef22fa7f87f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.814235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0e837-294c-4f39-94bd-eef22fa7f87f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.814517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0e837-294c-4f39-94bd-eef22fa7f87f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:17.825470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0e837-294c-4f39-94bd-eef22fa7f87f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:17.826356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f3b9e-0d37-4794-a96f-a282c5874522 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-57c0e837-294c-4f39-94bd-eef22fa7f87f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:17.830392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:17.831381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 318/624] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:17.837314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0319d0-d0c2-477a-9712-28fc494bb226 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.839126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0319d0-d0c2-477a-9712-28fc494bb226 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.839126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0319d0-d0c2-477a-9712-28fc494bb226 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:17.839126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 319/625] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:17.839126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0319d0-d0c2-477a-9712-28fc494bb226 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:17.869303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639cb65-9b3d-49e0-8754-595f3ea79e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:17.870017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639cb65-9b3d-49e0-8754-595f3ea79e6b None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.870292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639cb65-9b3d-49e0-8754-595f3ea79e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.870569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639cb65-9b3d-49e0-8754-595f3ea79e6b None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/b723681faf15434e8529e8fc56c6e9b8/roles/c6689cf082df4be9b69f2208c097f2c3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:17.887740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639cb65-9b3d-49e0-8754-595f3ea79e6b None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=b723681faf15434e8529e8fc56c6e9b8, role_id=c6689cf082df4be9b69f2208c097f2c3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:17.889337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639cb65-9b3d-49e0-8754-595f3ea79e6b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:17.943208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0319d0-d0c2-477a-9712-28fc494bb226 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:17.945975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:17.974409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 307/626] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:13:17 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/b723681faf15434e8529e8fc56c6e9b8/roles/c6689cf082df4be9b69f2208c097f2c3 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:13:17.975242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639cb65-9b3d-49e0-8754-595f3ea79e6b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:17.989047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f151985-a0bc-42fe-9d46-019a542fbc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:17.990144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f151985-a0bc-42fe-9d46-019a542fbc7e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:17.990624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f151985-a0bc-42fe-9d46-019a542fbc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:17.991128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f151985-a0bc-42fe-9d46-019a542fbc7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.005563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f151985-a0bc-42fe-9d46-019a542fbc7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-2f151985-a0bc-42fe-9d46-019a542fbc7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:18.011264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:18.012510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:18.013576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:18.015700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 320/627] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:18.017786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728656ac-6ef8-4d3b-aec6-d1bd59c1d5e2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.018246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728656ac-6ef8-4d3b-aec6-d1bd59c1d5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.018652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728656ac-6ef8-4d3b-aec6-d1bd59c1d5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.039593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-728656ac-6ef8-4d3b-aec6-d1bd59c1d5e2 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:18.041347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-728656ac-6ef8-4d3b-aec6-d1bd59c1d5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:18.056752 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:18.128170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 321/628] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:18.128170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728656ac-6ef8-4d3b-aec6-d1bd59c1d5e2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:18.145879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639cb65-9b3d-49e0-8754-595f3ea79e6b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:18.186889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c628f7-2e5e-4aa5-9c47-465820f4c387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:18.187510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c628f7-2e5e-4aa5-9c47-465820f4c387 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.188807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c628f7-2e5e-4aa5-9c47-465820f4c387 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.188807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c628f7-2e5e-4aa5-9c47-465820f4c387 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.188969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c628f7-2e5e-4aa5-9c47-465820f4c387 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:18.190184 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c628f7-2e5e-4aa5-9c47-465820f4c387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:18.191584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c628f7-2e5e-4aa5-9c47-465820f4c387 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:18.204546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 308/629] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:18.209962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c628f7-2e5e-4aa5-9c47-465820f4c387 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:18.266268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728656ac-6ef8-4d3b-aec6-d1bd59c1d5e2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:18.277817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:18.306250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c826ae-aa51-44e0-a574-9971651f350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:18.306926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c826ae-aa51-44e0-a574-9971651f350e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.307309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c826ae-aa51-44e0-a574-9971651f350e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.307635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c826ae-aa51-44e0-a574-9971651f350e None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.308311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c826ae-aa51-44e0-a574-9971651f350e None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:18.308618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c826ae-aa51-44e0-a574-9971651f350e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:18.310291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c826ae-aa51-44e0-a574-9971651f350e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:18.320295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 322/630] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:18.321407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c628f7-2e5e-4aa5-9c47-465820f4c387 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:18.326129 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c826ae-aa51-44e0-a574-9971651f350e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:18.332539 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:18.374743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceec112-5cbb-40bf-8f22-68f93d594af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:18.375389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceec112-5cbb-40bf-8f22-68f93d594af1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.375676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceec112-5cbb-40bf-8f22-68f93d594af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.376665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceec112-5cbb-40bf-8f22-68f93d594af1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.406627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceec112-5cbb-40bf-8f22-68f93d594af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:18.408603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceec112-5cbb-40bf-8f22-68f93d594af1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-bceec112-5cbb-40bf-8f22-68f93d594af1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:18.440325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:18.441560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:18.442547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:18.442547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:18.442547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:18.442547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 309/631] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:18.449717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceec112-5cbb-40bf-8f22-68f93d594af1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:18.459453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c826ae-aa51-44e0-a574-9971651f350e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:18.494845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db62d17-df74-4fba-843a-679286ca5fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:18.495484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db62d17-df74-4fba-843a-679286ca5fbb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.495765 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db62d17-df74-4fba-843a-679286ca5fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.496030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db62d17-df74-4fba-843a-679286ca5fbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.507603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db62d17-df74-4fba-843a-679286ca5fbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:18.509051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db62d17-df74-4fba-843a-679286ca5fbb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9db62d17-df74-4fba-843a-679286ca5fbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:18.524155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:18.525144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:18.526532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:18.526532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:18.526532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:18.526532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 323/632] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:18.527024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118be0a9-53ed-4ab8-aed4-39012b07cc5d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.527184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118be0a9-53ed-4ab8-aed4-39012b07cc5d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.527442 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118be0a9-53ed-4ab8-aed4-39012b07cc5d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.533005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 324/633] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:18.533005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118be0a9-53ed-4ab8-aed4-39012b07cc5d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:18.539798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:18.566628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceec112-5cbb-40bf-8f22-68f93d594af1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:18.607291 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3d105-2f19-4555-b150-30e4219ce373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:18.607925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3d105-2f19-4555-b150-30e4219ce373 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.608203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3d105-2f19-4555-b150-30e4219ce373 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.608473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3d105-2f19-4555-b150-30e4219ce373 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.609114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3d105-2f19-4555-b150-30e4219ce373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:18.609422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3d105-2f19-4555-b150-30e4219ce373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:18.611465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3d105-2f19-4555-b150-30e4219ce373 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:18.625157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 310/634] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:18.626762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3d105-2f19-4555-b150-30e4219ce373 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:18.640457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118be0a9-53ed-4ab8-aed4-39012b07cc5d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:18.653682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:18.677268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be5486-2564-4bc2-b9f3-bf251c8a3ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:18.677866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be5486-2564-4bc2-b9f3-bf251c8a3ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.678152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be5486-2564-4bc2-b9f3-bf251c8a3ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.678420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be5486-2564-4bc2-b9f3-bf251c8a3ad7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.689007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be5486-2564-4bc2-b9f3-bf251c8a3ad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-92be5486-2564-4bc2-b9f3-bf251c8a3ad7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:18.694239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:18.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:18.696033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 325/635] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:18.696739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7591c-20b1-4c93-afb9-802e2f0aec23 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.696739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7591c-20b1-4c93-afb9-802e2f0aec23 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.697356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7591c-20b1-4c93-afb9-802e2f0aec23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.716542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-40f7591c-20b1-4c93-afb9-802e2f0aec23 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:18.717770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-40f7591c-20b1-4c93-afb9-802e2f0aec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:18.753880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3d105-2f19-4555-b150-30e4219ce373 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:18.785941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:18.802795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 326/636] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:18.802981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd90ad7-12e5-4e33-a976-8881a6540a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:18.803114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7591c-20b1-4c93-afb9-802e2f0aec23 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:18.803208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd90ad7-12e5-4e33-a976-8881a6540a57 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.803208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd90ad7-12e5-4e33-a976-8881a6540a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.803208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd90ad7-12e5-4e33-a976-8881a6540a57 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.803208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd90ad7-12e5-4e33-a976-8881a6540a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:18.803208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd90ad7-12e5-4e33-a976-8881a6540a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:18.803208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd90ad7-12e5-4e33-a976-8881a6540a57 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:18.809774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 311/637] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:18.810718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd90ad7-12e5-4e33-a976-8881a6540a57 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:18.897836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:18.920499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd90ad7-12e5-4e33-a976-8881a6540a57 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:18.933242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7591c-20b1-4c93-afb9-802e2f0aec23 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:18.956507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32573dcd-cc89-46f3-9db8-89069aeaad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:18.957294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32573dcd-cc89-46f3-9db8-89069aeaad96 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.965301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32573dcd-cc89-46f3-9db8-89069aeaad96 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.965301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32573dcd-cc89-46f3-9db8-89069aeaad96 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.965301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32573dcd-cc89-46f3-9db8-89069aeaad96 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:18.965301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32573dcd-cc89-46f3-9db8-89069aeaad96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:18.965301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32573dcd-cc89-46f3-9db8-89069aeaad96 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:18.977596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 312/638] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:18.978667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32573dcd-cc89-46f3-9db8-89069aeaad96 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:18.983225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4449b5-3a1e-4293-bc2e-c72ba5183075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:18.984327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4449b5-3a1e-4293-bc2e-c72ba5183075 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:18.985815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4449b5-3a1e-4293-bc2e-c72ba5183075 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:18.986271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4449b5-3a1e-4293-bc2e-c72ba5183075 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:18.995379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4449b5-3a1e-4293-bc2e-c72ba5183075 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:18.997514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4449b5-3a1e-4293-bc2e-c72ba5183075 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6f4449b5-3a1e-4293-bc2e-c72ba5183075 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:19.014260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:19.015267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:19.016222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:19.016222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:19.016222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:19.016222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:19.016222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:19.016222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:19.016222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:19.016222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:19.016222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:13:19.016222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:19.023588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 327/639] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:19.023588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4449b5-3a1e-4293-bc2e-c72ba5183075 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:19.085183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:19.109702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32573dcd-cc89-46f3-9db8-89069aeaad96 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:19.131870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4449b5-3a1e-4293-bc2e-c72ba5183075 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:19.147032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22493f3f-f39f-429f-af00-eff45ff530b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:19.148350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22493f3f-f39f-429f-af00-eff45ff530b1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:19.148770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22493f3f-f39f-429f-af00-eff45ff530b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:19.149193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22493f3f-f39f-429f-af00-eff45ff530b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:19.156739 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22493f3f-f39f-429f-af00-eff45ff530b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:19.158816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22493f3f-f39f-429f-af00-eff45ff530b1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:19.161491 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:19.169371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb69f93-e2e0-4cbb-984b-d57c1588a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:19.170036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb69f93-e2e0-4cbb-984b-d57c1588a1ef None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:19.176642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb69f93-e2e0-4cbb-984b-d57c1588a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-22493f3f-f39f-429f-af00-eff45ff530b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:19.176774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:19.177768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:19.178728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:19.178728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:19.178728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:19.178728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:19.178728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:19.178728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:19.178728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:19.178728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:19.178728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:19.178728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:19.179270 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb69f93-e2e0-4cbb-984b-d57c1588a1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:19.179342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 313/640] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 1 05:13:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:19.179342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22493f3f-f39f-429f-af00-eff45ff530b1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:19.187360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb69f93-e2e0-4cbb-984b-d57c1588a1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:19.189343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb69f93-e2e0-4cbb-984b-d57c1588a1ef None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8fb69f93-e2e0-4cbb-984b-d57c1588a1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:19.204531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:19.205817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:19.207030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:19.207030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:19.207030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:19.210610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 328/641] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:19.211303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb69f93-e2e0-4cbb-984b-d57c1588a1ef None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:19.291179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22493f3f-f39f-429f-af00-eff45ff530b1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:19.322585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb69f93-e2e0-4cbb-984b-d57c1588a1ef None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:19.328101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321f1b4-fe3a-40c6-bcc2-ac5433c1f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:19.328835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f1b4-fe3a-40c6-bcc2-ac5433c1f6af None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:19.329216 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f1b4-fe3a-40c6-bcc2-ac5433c1f6af None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:19.329620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f1b4-fe3a-40c6-bcc2-ac5433c1f6af None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:19.330470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321f1b4-fe3a-40c6-bcc2-ac5433c1f6af None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:19.330880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321f1b4-fe3a-40c6-bcc2-ac5433c1f6af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:19.332413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:19.332539 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321f1b4-fe3a-40c6-bcc2-ac5433c1f6af None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:19.351022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 314/642] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 1 05:13:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:19.351765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321f1b4-fe3a-40c6-bcc2-ac5433c1f6af None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:19.362054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bf49d-84f1-4574-8e9c-32384756d1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:19.363390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bf49d-84f1-4574-8e9c-32384756d1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:19.364115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bf49d-84f1-4574-8e9c-32384756d1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:19.364393 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bf49d-84f1-4574-8e9c-32384756d1b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:19.374573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bf49d-84f1-4574-8e9c-32384756d1b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6c6bf49d-84f1-4574-8e9c-32384756d1b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:19.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:19.380682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 329/643] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:19.381669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bf49d-84f1-4574-8e9c-32384756d1b8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:19.416938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:19.466017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321f1b4-fe3a-40c6-bcc2-ac5433c1f6af None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:19.489929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bf49d-84f1-4574-8e9c-32384756d1b8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:19.503209 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12158c03-3346-430d-93ca-dc7bbfe99f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:19.503813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12158c03-3346-430d-93ca-dc7bbfe99f1d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:19.504086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12158c03-3346-430d-93ca-dc7bbfe99f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:19.504356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12158c03-3346-430d-93ca-dc7bbfe99f1d None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:19.505079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12158c03-3346-430d-93ca-dc7bbfe99f1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:19.505380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12158c03-3346-430d-93ca-dc7bbfe99f1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:19.506835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12158c03-3346-430d-93ca-dc7bbfe99f1d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:19.526521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a17f9a-679a-43a4-9ce2-6eb9f31306b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:19.531536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a17f9a-679a-43a4-9ce2-6eb9f31306b3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:19.531666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 315/644] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 1 05:13:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:19.531782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a17f9a-679a-43a4-9ce2-6eb9f31306b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:19.531782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a17f9a-679a-43a4-9ce2-6eb9f31306b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:19.532004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12158c03-3346-430d-93ca-dc7bbfe99f1d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:19.532097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a17f9a-679a-43a4-9ce2-6eb9f31306b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:19.532097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a17f9a-679a-43a4-9ce2-6eb9f31306b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:19.532097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a17f9a-679a-43a4-9ce2-6eb9f31306b3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:19.542900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 330/645] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:19.543794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a17f9a-679a-43a4-9ce2-6eb9f31306b3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:19.577541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:19.637121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12158c03-3346-430d-93ca-dc7bbfe99f1d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:19.655037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:19.657580 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a17f9a-679a-43a4-9ce2-6eb9f31306b3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:19.675851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c945c3-1807-4e6b-ba07-7e6216614801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:19.676470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c945c3-1807-4e6b-ba07-7e6216614801 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:19.676753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c945c3-1807-4e6b-ba07-7e6216614801 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:19.677026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c945c3-1807-4e6b-ba07-7e6216614801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:19.694708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0928d-a1ec-4c4e-aad1-b6690f10757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:19.695402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0928d-a1ec-4c4e-aad1-b6690f10757a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:19.695817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0928d-a1ec-4c4e-aad1-b6690f10757a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:19.696154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0928d-a1ec-4c4e-aad1-b6690f10757a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:19.696997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0928d-a1ec-4c4e-aad1-b6690f10757a None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:19.697370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0928d-a1ec-4c4e-aad1-b6690f10757a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:19.699087 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0928d-a1ec-4c4e-aad1-b6690f10757a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:19.713159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 331/646] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:19.713159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0928d-a1ec-4c4e-aad1-b6690f10757a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:19.756167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c945c3-1807-4e6b-ba07-7e6216614801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:19.757651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c945c3-1807-4e6b-ba07-7e6216614801 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:19.816344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:19.827284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0928d-a1ec-4c4e-aad1-b6690f10757a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:19.863528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 316/647] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:19.864850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb1445-f2d5-47ce-a177-38cdf5eae5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:19.865482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb1445-f2d5-47ce-a177-38cdf5eae5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:19.865772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb1445-f2d5-47ce-a177-38cdf5eae5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:19.866044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb1445-f2d5-47ce-a177-38cdf5eae5f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:19.868854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c945c3-1807-4e6b-ba07-7e6216614801 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:19.887563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:19.939660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb1445-f2d5-47ce-a177-38cdf5eae5f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:19.941287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb1445-f2d5-47ce-a177-38cdf5eae5f6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:19.981523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c945c3-1807-4e6b-ba07-7e6216614801 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:20.019936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48f135-2d19-4996-99a0-ac78deac76f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:20.020566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48f135-2d19-4996-99a0-ac78deac76f3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.020844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48f135-2d19-4996-99a0-ac78deac76f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.021116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48f135-2d19-4996-99a0-ac78deac76f3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.028611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48f135-2d19-4996-99a0-ac78deac76f3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:20.030102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48f135-2d19-4996-99a0-ac78deac76f3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:20.040871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-ef48f135-2d19-4996-99a0-ac78deac76f3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:20.042438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:20.043364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 317/648] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:20.044321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48f135-2d19-4996-99a0-ac78deac76f3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.056410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 332/649] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:20.057162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb1445-f2d5-47ce-a177-38cdf5eae5f6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.130203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:20.153014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48f135-2d19-4996-99a0-ac78deac76f3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:20.158724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb1445-f2d5-47ce-a177-38cdf5eae5f6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:20.188760 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f66b74-519c-468f-8537-cd5bc057b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:20.189533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f66b74-519c-468f-8537-cd5bc057b34e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.193896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f66b74-519c-468f-8537-cd5bc057b34e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.193896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f66b74-519c-468f-8537-cd5bc057b34e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.194082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28125c5d-64ac-4ee6-b45c-cabf3b485744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:20.194082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28125c5d-64ac-4ee6-b45c-cabf3b485744 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.194082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28125c5d-64ac-4ee6-b45c-cabf3b485744 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.194082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28125c5d-64ac-4ee6-b45c-cabf3b485744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.201060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f66b74-519c-468f-8537-cd5bc057b34e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:20.202999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f66b74-519c-468f-8537-cd5bc057b34e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:20.203545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28125c5d-64ac-4ee6-b45c-cabf3b485744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:20.205667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28125c5d-64ac-4ee6-b45c-cabf3b485744 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-18f66b74-519c-468f-8537-cd5bc057b34e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:20.217672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:20.218755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:20.220276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:20.220276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:20.220276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:20.220276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 318/650] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:20.220276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f66b74-519c-468f-8537-cd5bc057b34e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-28125c5d-64ac-4ee6-b45c-cabf3b485744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:20.221148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:20.222137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:20.224316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:20.224316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:20.224316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:20.224316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 333/651] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:20.224316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28125c5d-64ac-4ee6-b45c-cabf3b485744 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.267910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:20.324676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28125c5d-64ac-4ee6-b45c-cabf3b485744 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:20.325335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f66b74-519c-468f-8537-cd5bc057b34e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:20.350289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:20.357006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce30dab-7d11-430e-9764-ddc9c559f08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:20.357578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9c330-2b3e-48ab-9a7c-916155985115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:20.357705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce30dab-7d11-430e-9764-ddc9c559f08c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.359939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce30dab-7d11-430e-9764-ddc9c559f08c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.360592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9c330-2b3e-48ab-9a7c-916155985115 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.360744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce30dab-7d11-430e-9764-ddc9c559f08c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.360841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9c330-2b3e-48ab-9a7c-916155985115 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.360841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9c330-2b3e-48ab-9a7c-916155985115 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.360841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9c330-2b3e-48ab-9a7c-916155985115 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:20.360841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9c330-2b3e-48ab-9a7c-916155985115 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:20.361819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9c330-2b3e-48ab-9a7c-916155985115 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:20.369949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce30dab-7d11-430e-9764-ddc9c559f08c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:20.371410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce30dab-7d11-430e-9764-ddc9c559f08c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:20.379663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 319/652] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:20.381126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31186dc1-3c3b-4163-9c08-669f2878316d None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.381644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31186dc1-3c3b-4163-9c08-669f2878316d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.382134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31186dc1-3c3b-4163-9c08-669f2878316d None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.383503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 320/653] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:20.384188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31186dc1-3c3b-4163-9c08-669f2878316d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-fce30dab-7d11-430e-9764-ddc9c559f08c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:20.390895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:20.392590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:20.393626 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:20.393626 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:20.393626 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:20.393626 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 334/654] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:20.393997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce30dab-7d11-430e-9764-ddc9c559f08c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.486082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:20.490695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31186dc1-3c3b-4163-9c08-669f2878316d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:20.492301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce30dab-7d11-430e-9764-ddc9c559f08c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:20.524985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d0c1c-e436-48d6-af36-e5219029e36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:20.525544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0c1c-e436-48d6-af36-e5219029e36e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.525822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0c1c-e436-48d6-af36-e5219029e36e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.526092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0c1c-e436-48d6-af36-e5219029e36e None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.526724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d0c1c-e436-48d6-af36-e5219029e36e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:20.527026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d0c1c-e436-48d6-af36-e5219029e36e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:20.527604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470b282-0745-44d3-a08f-e99253ad1f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:20.528213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b282-0745-44d3-a08f-e99253ad1f1d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.528402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d0c1c-e436-48d6-af36-e5219029e36e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:20.528495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b282-0745-44d3-a08f-e99253ad1f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.528755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b282-0745-44d3-a08f-e99253ad1f1d None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.529391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470b282-0745-44d3-a08f-e99253ad1f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:20.529733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470b282-0745-44d3-a08f-e99253ad1f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:20.531080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470b282-0745-44d3-a08f-e99253ad1f1d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:20.539369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 321/655] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:20.540159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d0c1c-e436-48d6-af36-e5219029e36e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.543747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 335/656] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:20.544952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63a028-811c-4deb-bf8e-428e9425bc38 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.545239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63a028-811c-4deb-bf8e-428e9425bc38 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.545538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63a028-811c-4deb-bf8e-428e9425bc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.562936 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fe63a028-811c-4deb-bf8e-428e9425bc38 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:20.564683 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fe63a028-811c-4deb-bf8e-428e9425bc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:20.588533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:20.631846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 336/657] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:20.632684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63a028-811c-4deb-bf8e-428e9425bc38 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.652712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d0c1c-e436-48d6-af36-e5219029e36e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:20.685856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bb6d3-9f5f-40c1-82c1-2011b5a80261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:20.686457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bb6d3-9f5f-40c1-82c1-2011b5a80261 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.686731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bb6d3-9f5f-40c1-82c1-2011b5a80261 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.687066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bb6d3-9f5f-40c1-82c1-2011b5a80261 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.721711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:20.737778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63a028-811c-4deb-bf8e-428e9425bc38 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:20.767254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bb6d3-9f5f-40c1-82c1-2011b5a80261 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:20.768855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bb6d3-9f5f-40c1-82c1-2011b5a80261 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:20.774979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf82a7-67f2-469c-abad-efda58516681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:20.775649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf82a7-67f2-469c-abad-efda58516681 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.775869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf82a7-67f2-469c-abad-efda58516681 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.776141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf82a7-67f2-469c-abad-efda58516681 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.776898 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf82a7-67f2-469c-abad-efda58516681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:20.777086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf82a7-67f2-469c-abad-efda58516681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:20.778471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf82a7-67f2-469c-abad-efda58516681 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:20.795000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 337/658] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:20.795000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf82a7-67f2-469c-abad-efda58516681 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.823788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:20.885770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 322/659] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:20.887535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bb6d3-9f5f-40c1-82c1-2011b5a80261 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.899309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf82a7-67f2-469c-abad-efda58516681 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:20.933733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887ebeb-f4f4-43b1-a5e6-31f3af0c1188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:20.934362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887ebeb-f4f4-43b1-a5e6-31f3af0c1188 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:20.934636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887ebeb-f4f4-43b1-a5e6-31f3af0c1188 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:20.934907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887ebeb-f4f4-43b1-a5e6-31f3af0c1188 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:20.942872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887ebeb-f4f4-43b1-a5e6-31f3af0c1188 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:20.944298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887ebeb-f4f4-43b1-a5e6-31f3af0c1188 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-b887ebeb-f4f4-43b1-a5e6-31f3af0c1188 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:20.955482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:20.956464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 338/660] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:20.957481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887ebeb-f4f4-43b1-a5e6-31f3af0c1188 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:20.961021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:20.989715 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bb6d3-9f5f-40c1-82c1-2011b5a80261 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.031195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf81c6e-9d47-4b52-917f-d87468a59b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.031738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81c6e-9d47-4b52-917f-d87468a59b3c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.032148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81c6e-9d47-4b52-917f-d87468a59b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.032477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81c6e-9d47-4b52-917f-d87468a59b3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.039124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf81c6e-9d47-4b52-917f-d87468a59b3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:21.040556 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf81c6e-9d47-4b52-917f-d87468a59b3c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-3bf81c6e-9d47-4b52-917f-d87468a59b3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:21.052234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:21.053121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:21.054070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:21.054070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:21.054070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:21.054070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:21.054070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:21.054070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:21.054070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:21.054070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:21.054070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 323/661] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:21.054070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf81c6e-9d47-4b52-917f-d87468a59b3c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.062004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:21.063978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887ebeb-f4f4-43b1-a5e6-31f3af0c1188 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.097864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d2f18-7aa0-45c7-af20-48e8decccfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.098487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d2f18-7aa0-45c7-af20-48e8decccfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.098763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d2f18-7aa0-45c7-af20-48e8decccfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.099034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d2f18-7aa0-45c7-af20-48e8decccfe8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.105638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d2f18-7aa0-45c7-af20-48e8decccfe8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:21.107063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d2f18-7aa0-45c7-af20-48e8decccfe8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-b11d2f18-7aa0-45c7-af20-48e8decccfe8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:21.118549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:21.119588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:21.120631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:21.120631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:21.120631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:21.120631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:21.120631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:21.120631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:21.120631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:21.120631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:21.120631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 339/662] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:21.120631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d2f18-7aa0-45c7-af20-48e8decccfe8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.159773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf81c6e-9d47-4b52-917f-d87468a59b3c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.183167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:21.192257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140efbe-4db7-42fd-97cc-2f1e270db3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.192875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140efbe-4db7-42fd-97cc-2f1e270db3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.195107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140efbe-4db7-42fd-97cc-2f1e270db3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.195107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140efbe-4db7-42fd-97cc-2f1e270db3f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.205130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140efbe-4db7-42fd-97cc-2f1e270db3f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:21.206743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140efbe-4db7-42fd-97cc-2f1e270db3f7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-7140efbe-4db7-42fd-97cc-2f1e270db3f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:21.222096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:21.226269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:21.227196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:21.227196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:21.227196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:21.227372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d2f18-7aa0-45c7-af20-48e8decccfe8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.227456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 324/663] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:21.227456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140efbe-4db7-42fd-97cc-2f1e270db3f7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.260006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273b347-983b-4efd-b931-bc09dcaca627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.260627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273b347-983b-4efd-b931-bc09dcaca627 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.260899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273b347-983b-4efd-b931-bc09dcaca627 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.261164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273b347-983b-4efd-b931-bc09dcaca627 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.271390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273b347-983b-4efd-b931-bc09dcaca627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3273b347-983b-4efd-b931-bc09dcaca627 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:21.276565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:21.277633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:21.278664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 340/664] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:21.280352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273b347-983b-4efd-b931-bc09dcaca627 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.284105 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:21.326852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140efbe-4db7-42fd-97cc-2f1e270db3f7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.361244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9944d-14ff-4311-bab4-f553f28d2d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.361881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9944d-14ff-4311-bab4-f553f28d2d4f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.362165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9944d-14ff-4311-bab4-f553f28d2d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.362440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9944d-14ff-4311-bab4-f553f28d2d4f None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.363165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9944d-14ff-4311-bab4-f553f28d2d4f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:21.363475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9944d-14ff-4311-bab4-f553f28d2d4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:21.364902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9944d-14ff-4311-bab4-f553f28d2d4f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:21.378162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273b347-983b-4efd-b931-bc09dcaca627 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.383015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 325/665] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:21.383642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9944d-14ff-4311-bab4-f553f28d2d4f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.404131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:21.414392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901716e-42f7-4dba-a75d-367c53c28a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.414992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901716e-42f7-4dba-a75d-367c53c28a67 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.415266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901716e-42f7-4dba-a75d-367c53c28a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.415563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901716e-42f7-4dba-a75d-367c53c28a67 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.416239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901716e-42f7-4dba-a75d-367c53c28a67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:21.416538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901716e-42f7-4dba-a75d-367c53c28a67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:21.417986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901716e-42f7-4dba-a75d-367c53c28a67 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:21.429229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 341/666] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:21.429965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901716e-42f7-4dba-a75d-367c53c28a67 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.486416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9944d-14ff-4311-bab4-f553f28d2d4f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.503223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:21.521870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a3251-5841-4474-b150-d2a1ba3149f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.522648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a3251-5841-4474-b150-d2a1ba3149f9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.523138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a3251-5841-4474-b150-d2a1ba3149f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.523491 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a3251-5841-4474-b150-d2a1ba3149f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.524085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a3251-5841-4474-b150-d2a1ba3149f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:21.524394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a3251-5841-4474-b150-d2a1ba3149f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:21.525895 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a3251-5841-4474-b150-d2a1ba3149f9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:21.529778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901716e-42f7-4dba-a75d-367c53c28a67 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.537436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 326/667] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:21.538211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a3251-5841-4474-b150-d2a1ba3149f9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.567790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01fbec-1c9b-4926-a6ee-28f046a1aeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.568375 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01fbec-1c9b-4926-a6ee-28f046a1aeef None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.568649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01fbec-1c9b-4926-a6ee-28f046a1aeef None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.568914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01fbec-1c9b-4926-a6ee-28f046a1aeef None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.569574 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01fbec-1c9b-4926-a6ee-28f046a1aeef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:21.569873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01fbec-1c9b-4926-a6ee-28f046a1aeef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:21.571213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01fbec-1c9b-4926-a6ee-28f046a1aeef None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:21.583142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 342/668] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:21.583923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01fbec-1c9b-4926-a6ee-28f046a1aeef None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.628047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:21.640598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a3251-5841-4474-b150-d2a1ba3149f9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.676070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbbbac-2d01-4007-9db7-dbc955268ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.676774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbbbac-2d01-4007-9db7-dbc955268ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.677056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbbbac-2d01-4007-9db7-dbc955268ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.677337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbbbac-2d01-4007-9db7-dbc955268ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.678032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbbbac-2d01-4007-9db7-dbc955268ac2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:21.678338 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbbbac-2d01-4007-9db7-dbc955268ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:21.679716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbbbac-2d01-4007-9db7-dbc955268ac2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:21.685613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01fbec-1c9b-4926-a6ee-28f046a1aeef None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.691439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 327/669] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:21.691439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbbbac-2d01-4007-9db7-dbc955268ac2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.724481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b2846-7992-4c3e-926e-09a0c003d500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.725082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b2846-7992-4c3e-926e-09a0c003d500 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.725355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b2846-7992-4c3e-926e-09a0c003d500 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.725654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b2846-7992-4c3e-926e-09a0c003d500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.734631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:21.793922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b2846-7992-4c3e-926e-09a0c003d500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:21.795319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b2846-7992-4c3e-926e-09a0c003d500 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:21.795470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbbbac-2d01-4007-9db7-dbc955268ac2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.830568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d7e3c-3d62-4b40-b322-e55f9f7fde1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:21.831174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d7e3c-3d62-4b40-b322-e55f9f7fde1f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:21.831456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d7e3c-3d62-4b40-b322-e55f9f7fde1f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:21.831727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d7e3c-3d62-4b40-b322-e55f9f7fde1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:21.832451 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d7e3c-3d62-4b40-b322-e55f9f7fde1f None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=4c23597e0da2454abd2a5354d779b092) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:21.832762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d7e3c-3d62-4b40-b322-e55f9f7fde1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:21.834226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d7e3c-3d62-4b40-b322-e55f9f7fde1f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:21.839964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 328/670] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/role_assignments?user.id=4c23597e0da2454abd2a5354d779b092&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:21.840738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d7e3c-3d62-4b40-b322-e55f9f7fde1f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.840860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:21.909238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 343/671] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:21.910513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b2846-7992-4c3e-926e-09a0c003d500 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:21.967316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d7e3c-3d62-4b40-b322-e55f9f7fde1f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:21.969158 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:22.011955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0fe1d-6a31-4f9e-b166-13a492fcf14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:22.012930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0fe1d-6a31-4f9e-b166-13a492fcf14a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:22.012930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0fe1d-6a31-4f9e-b166-13a492fcf14a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:22.013112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0fe1d-6a31-4f9e-b166-13a492fcf14a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:22.013934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0fe1d-6a31-4f9e-b166-13a492fcf14a None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=7cda3c1f67e54ee08abc014399b07cc5) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:22.014160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0fe1d-6a31-4f9e-b166-13a492fcf14a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:22.015600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0fe1d-6a31-4f9e-b166-13a492fcf14a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:22.022718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 329/672] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/role_assignments?user.id=7cda3c1f67e54ee08abc014399b07cc5&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:22.023610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0fe1d-6a31-4f9e-b166-13a492fcf14a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:22.035623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b2846-7992-4c3e-926e-09a0c003d500 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:22.087483 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcc0a3-54ad-4a94-a9a5-392d6fad8085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:22.088619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcc0a3-54ad-4a94-a9a5-392d6fad8085 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:22.088998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcc0a3-54ad-4a94-a9a5-392d6fad8085 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:22.089293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcc0a3-54ad-4a94-a9a5-392d6fad8085 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:22.097563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfcc0a3-54ad-4a94-a9a5-392d6fad8085 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:22.099052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfcc0a3-54ad-4a94-a9a5-392d6fad8085 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8dfcc0a3-54ad-4a94-a9a5-392d6fad8085 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:22.113242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:22.114468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:22.115514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:22.115514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:22.115514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:22.115514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:22.115514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:22.115514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:22.115514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:22.115514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:22.115514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 344/673] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:22.115514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcc0a3-54ad-4a94-a9a5-392d6fad8085 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:22.133543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:22.175984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0fe1d-6a31-4f9e-b166-13a492fcf14a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:22.226127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e9749-e9d9-4ff2-87ef-2d0abc6f85e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:22.227114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9749-e9d9-4ff2-87ef-2d0abc6f85e1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:22.227396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9749-e9d9-4ff2-87ef-2d0abc6f85e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:22.227671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9749-e9d9-4ff2-87ef-2d0abc6f85e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:22.240694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e9749-e9d9-4ff2-87ef-2d0abc6f85e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:22.242582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e9749-e9d9-4ff2-87ef-2d0abc6f85e1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:22.245836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcc0a3-54ad-4a94-a9a5-392d6fad8085 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-de8e9749-e9d9-4ff2-87ef-2d0abc6f85e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:22.260686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:22.261754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:22.262661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:22.262661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:22.262661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:22.263282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 330/674] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:22.264349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e9749-e9d9-4ff2-87ef-2d0abc6f85e1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:22.289155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c8e3b-2120-42e1-bc3e-6a31ad994071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:22.289824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c8e3b-2120-42e1-bc3e-6a31ad994071 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:22.290092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c8e3b-2120-42e1-bc3e-6a31ad994071 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:22.290394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c8e3b-2120-42e1-bc3e-6a31ad994071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:22.293486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:22.301066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c8e3b-2120-42e1-bc3e-6a31ad994071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:22.302688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c8e3b-2120-42e1-bc3e-6a31ad994071 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-308c8e3b-2120-42e1-bc3e-6a31ad994071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:22.320737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:22.321846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:22.322752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:22.322752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:22.322752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:22.322752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 345/675] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:22.323724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c8e3b-2120-42e1-bc3e-6a31ad994071 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:22.395559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:22.414596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e9749-e9d9-4ff2-87ef-2d0abc6f85e1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:22.444594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c8e3b-2120-42e1-bc3e-6a31ad994071 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:22.457213 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d80ac-4f78-4424-8de3-318217b61a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:22.457875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d80ac-4f78-4424-8de3-318217b61a4b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:22.458163 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d80ac-4f78-4424-8de3-318217b61a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:22.458460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d80ac-4f78-4424-8de3-318217b61a4b None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:22.459149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d80ac-4f78-4424-8de3-318217b61a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:22.459462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d80ac-4f78-4424-8de3-318217b61a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:22.461021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d80ac-4f78-4424-8de3-318217b61a4b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:22.479027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 331/676] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:22.480011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d80ac-4f78-4424-8de3-318217b61a4b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:22.483127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d81ca-6660-40f6-b92b-4343024f1a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:22.483929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d81ca-6660-40f6-b92b-4343024f1a6f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:22.484209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d81ca-6660-40f6-b92b-4343024f1a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:22.484484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d81ca-6660-40f6-b92b-4343024f1a6f None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:22.485757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d81ca-6660-40f6-b92b-4343024f1a6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:22.486067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d81ca-6660-40f6-b92b-4343024f1a6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:22.487429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d81ca-6660-40f6-b92b-4343024f1a6f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:22.500328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 346/677] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:22.501813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d81ca-6660-40f6-b92b-4343024f1a6f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:22.585297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:22.606322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d81ca-6660-40f6-b92b-4343024f1a6f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:22.619435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d80ac-4f78-4424-8de3-318217b61a4b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:22.632865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:22.656835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c47f5-c924-4c99-b2d8-ffc20659797f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:22.658410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c47f5-c924-4c99-b2d8-ffc20659797f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:22.658846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c47f5-c924-4c99-b2d8-ffc20659797f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:22.659300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c47f5-c924-4c99-b2d8-ffc20659797f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:22.660490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c47f5-c924-4c99-b2d8-ffc20659797f None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=7e0e8589931b46ca84168a2ad273b63d) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:22.661495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c47f5-c924-4c99-b2d8-ffc20659797f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:22.663691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c47f5-c924-4c99-b2d8-ffc20659797f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:22.670282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41119d0-9cf8-4097-8171-a27d88906abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:22.671029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41119d0-9cf8-4097-8171-a27d88906abd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:22.671463 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41119d0-9cf8-4097-8171-a27d88906abd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:22.671837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41119d0-9cf8-4097-8171-a27d88906abd None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:22.672972 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 347/678] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:22 2022] GET /identity/v3/role_assignments?user.id=7e0e8589931b46ca84168a2ad273b63d&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:22.673968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c47f5-c924-4c99-b2d8-ffc20659797f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:22.678580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41119d0-9cf8-4097-8171-a27d88906abd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:22.678580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41119d0-9cf8-4097-8171-a27d88906abd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:22.679782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41119d0-9cf8-4097-8171-a27d88906abd None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:22.708049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 332/679] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:22.716600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41119d0-9cf8-4097-8171-a27d88906abd None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:22.807897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c47f5-c924-4c99-b2d8-ffc20659797f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:22.851534 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b69b4-cd5b-479f-939e-61101eeb9926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:22.852466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b69b4-cd5b-479f-939e-61101eeb9926 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:22.853292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b69b4-cd5b-479f-939e-61101eeb9926 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:22.853773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b69b4-cd5b-479f-939e-61101eeb9926 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:22.863647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b69b4-cd5b-479f-939e-61101eeb9926 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:22.865828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b69b4-cd5b-479f-939e-61101eeb9926 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-fe0b69b4-cd5b-479f-939e-61101eeb9926 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:22.880416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:22.881692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:22.882879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:22.882879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:22.882879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:22.882879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:22.882879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:22.882879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:22.882879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:22.882879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:22.882879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:22.882879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:22.885475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 348/680] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:22.891907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b69b4-cd5b-479f-939e-61101eeb9926 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:22.945811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:22.956554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41119d0-9cf8-4097-8171-a27d88906abd None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:23.002127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd389e5-268d-498b-a0ef-ce46b62a297f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:23.002718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd389e5-268d-498b-a0ef-ce46b62a297f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:23.003099 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd389e5-268d-498b-a0ef-ce46b62a297f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:23.003617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd389e5-268d-498b-a0ef-ce46b62a297f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:23.012472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd389e5-268d-498b-a0ef-ce46b62a297f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:23.014311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd389e5-268d-498b-a0ef-ce46b62a297f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:23.017230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-3bd389e5-268d-498b-a0ef-ce46b62a297f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:23.034147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:23.035324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:23.036228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:23.036228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:23.036228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:23.036228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:23.036228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:23.036228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:23.036228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:23.036228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:23.036672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b69b4-cd5b-479f-939e-61101eeb9926 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:23.038897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 333/681] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:23.051028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd389e5-268d-498b-a0ef-ce46b62a297f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:23.083832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a6abe-8db7-4ff7-ba17-a95bb80ec732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:23.084802 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a6abe-8db7-4ff7-ba17-a95bb80ec732 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:23.085376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a6abe-8db7-4ff7-ba17-a95bb80ec732 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:23.085945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a6abe-8db7-4ff7-ba17-a95bb80ec732 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:23.086854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a6abe-8db7-4ff7-ba17-a95bb80ec732 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:23.087307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a6abe-8db7-4ff7-ba17-a95bb80ec732 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:23.089168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a6abe-8db7-4ff7-ba17-a95bb80ec732 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:23.112672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 349/682] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 1 05:13:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:23.118469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a6abe-8db7-4ff7-ba17-a95bb80ec732 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:23.183274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd389e5-268d-498b-a0ef-ce46b62a297f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:23.233174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cd31a-c60d-4fe7-904e-e358a4a192f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:23.235077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cd31a-c60d-4fe7-904e-e358a4a192f9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:23.236375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cd31a-c60d-4fe7-904e-e358a4a192f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:23.237066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cd31a-c60d-4fe7-904e-e358a4a192f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:23.239159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104cd31a-c60d-4fe7-904e-e358a4a192f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:23.241000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104cd31a-c60d-4fe7-904e-e358a4a192f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:23.241804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:23.242876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104cd31a-c60d-4fe7-904e-e358a4a192f9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:23.245616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a6abe-8db7-4ff7-ba17-a95bb80ec732 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:23.268410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 334/683] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:23.278551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cd31a-c60d-4fe7-904e-e358a4a192f9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:23.307641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a641d8a-8c9b-4cb5-b632-5ba16e66d765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:23.309563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a641d8a-8c9b-4cb5-b632-5ba16e66d765 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:23.309851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a641d8a-8c9b-4cb5-b632-5ba16e66d765 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:23.310125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a641d8a-8c9b-4cb5-b632-5ba16e66d765 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:23.391738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:23.432747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a641d8a-8c9b-4cb5-b632-5ba16e66d765 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:23.434382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a641d8a-8c9b-4cb5-b632-5ba16e66d765 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:23.471838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cd31a-c60d-4fe7-904e-e358a4a192f9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:23.529557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6ce5c-dc58-40e1-8dc7-7319be6d91af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:23.530158 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6ce5c-dc58-40e1-8dc7-7319be6d91af None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:23.530442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6ce5c-dc58-40e1-8dc7-7319be6d91af None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:23.530714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6ce5c-dc58-40e1-8dc7-7319be6d91af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:23.531455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6ce5c-dc58-40e1-8dc7-7319be6d91af None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=0e21910fca40479ca04e7f0418f7dd2d) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:23.532398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6ce5c-dc58-40e1-8dc7-7319be6d91af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:23.535485 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6ce5c-dc58-40e1-8dc7-7319be6d91af None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:23.551047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 335/684] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:23 2022] GET /identity/v3/role_assignments?user.id=0e21910fca40479ca04e7f0418f7dd2d&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:23.599010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:23.604642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 350/685] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:13:23.610453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6ce5c-dc58-40e1-8dc7-7319be6d91af None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:23.747673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6ce5c-dc58-40e1-8dc7-7319be6d91af None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:23.774613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:23.795029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abdc2b-2e21-4edf-82c0-3078eee7c147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:23.795029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abdc2b-2e21-4edf-82c0-3078eee7c147 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:23.795029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abdc2b-2e21-4edf-82c0-3078eee7c147 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:23.795029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abdc2b-2e21-4edf-82c0-3078eee7c147 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:23.814104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abdc2b-2e21-4edf-82c0-3078eee7c147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:23.815631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abdc2b-2e21-4edf-82c0-3078eee7c147 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-66abdc2b-2e21-4edf-82c0-3078eee7c147 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:23.835557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:23.838067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:23.839903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:23.839903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:23.839903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:23.839903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 336/686] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:23.843337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a641d8a-8c9b-4cb5-b632-5ba16e66d765 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:23.957959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a641d8a-8c9b-4cb5-b632-5ba16e66d765 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:23.995086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c548cf2-f8bc-4a78-b50f-bd926a6cc356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:23.996066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c548cf2-f8bc-4a78-b50f-bd926a6cc356 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:23.996559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c548cf2-f8bc-4a78-b50f-bd926a6cc356 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:23.997032 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c548cf2-f8bc-4a78-b50f-bd926a6cc356 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:23.998053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c548cf2-f8bc-4a78-b50f-bd926a6cc356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:23.998571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c548cf2-f8bc-4a78-b50f-bd926a6cc356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:24.000124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c548cf2-f8bc-4a78-b50f-bd926a6cc356 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:24.013332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 351/687] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:24.019257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abdc2b-2e21-4edf-82c0-3078eee7c147 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:24.100919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d9aaa-b90c-40df-b9d9-abd41850eb64 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:24.101582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d9aaa-b90c-40df-b9d9-abd41850eb64 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:24.102072 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d9aaa-b90c-40df-b9d9-abd41850eb64 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:24.104742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 352/688] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:24.108716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6efb8-86d6-421a-ad98-eae5a0cc3c37 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:24.109005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6efb8-86d6-421a-ad98-eae5a0cc3c37 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:24.109284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6efb8-86d6-421a-ad98-eae5a0cc3c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:24.140902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3ee6efb8-86d6-421a-ad98-eae5a0cc3c37 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:24.142821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3ee6efb8-86d6-421a-ad98-eae5a0cc3c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:24.161856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abdc2b-2e21-4edf-82c0-3078eee7c147 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:24.228229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27fba7-35a4-4d73-846d-ee1ff5e86077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:24.229611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27fba7-35a4-4d73-846d-ee1ff5e86077 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:24.230067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27fba7-35a4-4d73-846d-ee1ff5e86077 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:24.231383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27fba7-35a4-4d73-846d-ee1ff5e86077 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:24.241829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:24.246184 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27fba7-35a4-4d73-846d-ee1ff5e86077 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:24.247644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27fba7-35a4-4d73-846d-ee1ff5e86077 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-ac27fba7-35a4-4d73-846d-ee1ff5e86077 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:24.270817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:24.271776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:24.272723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:24.272723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:24.272723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:24.272723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:24.272723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:24.272723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:24.272723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:24.272723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:24.272723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 337/689] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:24.278023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27fba7-35a4-4d73-846d-ee1ff5e86077 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:24.283701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 353/690] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:13:24.286724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a916b-15d9-4ebf-8df6-1a43658b9fb8 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:24.287014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a916b-15d9-4ebf-8df6-1a43658b9fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:24.287283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a916b-15d9-4ebf-8df6-1a43658b9fb8 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:24.289594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 354/691] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:24.294008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83834b-c574-4441-b4e5-8db73507c58c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:24.294283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83834b-c574-4441-b4e5-8db73507c58c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:24.294556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83834b-c574-4441-b4e5-8db73507c58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:24.321147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:24.326531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4f83834b-c574-4441-b4e5-8db73507c58c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:24.327719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4f83834b-c574-4441-b4e5-8db73507c58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:24.445006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27fba7-35a4-4d73-846d-ee1ff5e86077 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:24.458349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 355/692] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:13:24.481185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d90cf-2843-4c21-9e58-a04efc8531d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:24.481820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d90cf-2843-4c21-9e58-a04efc8531d1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:24.482092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d90cf-2843-4c21-9e58-a04efc8531d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:24.482359 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d90cf-2843-4c21-9e58-a04efc8531d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:24.483038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d90cf-2843-4c21-9e58-a04efc8531d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:24.483336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d90cf-2843-4c21-9e58-a04efc8531d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:24.484827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d90cf-2843-4c21-9e58-a04efc8531d1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:24.497066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 338/693] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:24.503103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83834b-c574-4441-b4e5-8db73507c58c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:24.662193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83834b-c574-4441-b4e5-8db73507c58c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:24.695096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:24.713799 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fa1ca-a2d7-4a83-9768-182240f2a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:24.714410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fa1ca-a2d7-4a83-9768-182240f2a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:24.714687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fa1ca-a2d7-4a83-9768-182240f2a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:24.714956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fa1ca-a2d7-4a83-9768-182240f2a0b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:24.715686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fa1ca-a2d7-4a83-9768-182240f2a0b8 None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=b723681faf15434e8529e8fc56c6e9b8) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:24.715990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fa1ca-a2d7-4a83-9768-182240f2a0b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:24.717413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fa1ca-a2d7-4a83-9768-182240f2a0b8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:24.724448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 356/694] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:24 2022] GET /identity/v3/role_assignments?user.id=b723681faf15434e8529e8fc56c6e9b8&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:24.770762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d90cf-2843-4c21-9e58-a04efc8531d1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:24.801043 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fa1ca-a2d7-4a83-9768-182240f2a0b8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:24.917919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fa1ca-a2d7-4a83-9768-182240f2a0b8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:24.945180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:24.957065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d90cf-2843-4c21-9e58-a04efc8531d1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:24.958283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebe1d6-df89-429d-8b13-2f9c20875b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:24.958494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebe1d6-df89-429d-8b13-2f9c20875b87 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:24.958846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebe1d6-df89-429d-8b13-2f9c20875b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:24.959217 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebe1d6-df89-429d-8b13-2f9c20875b87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:24.971382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbebe1d6-df89-429d-8b13-2f9c20875b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-cbebe1d6-df89-429d-8b13-2f9c20875b87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:24.977014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:24.978263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:24.979180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:24.984100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 357/695] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:24.986009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebe1d6-df89-429d-8b13-2f9c20875b87 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:25.013233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d8dff-8918-4619-80fb-ebc7e108985f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:25.014303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d8dff-8918-4619-80fb-ebc7e108985f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:25.014806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d8dff-8918-4619-80fb-ebc7e108985f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:25.015304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d8dff-8918-4619-80fb-ebc7e108985f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:25.033739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:25.037626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d8dff-8918-4619-80fb-ebc7e108985f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:25.040283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d8dff-8918-4619-80fb-ebc7e108985f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-2f4d8dff-8918-4619-80fb-ebc7e108985f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:25.065050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:25.066136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:25.067073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:25.067073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:25.067073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:25.067073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 339/696] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:25.072648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d8dff-8918-4619-80fb-ebc7e108985f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:25.114115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebe1d6-df89-429d-8b13-2f9c20875b87 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:25.155327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fc35-4409-4080-baa7-a027646786ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:25.156305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fc35-4409-4080-baa7-a027646786ad None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:25.157204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fc35-4409-4080-baa7-a027646786ad None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:25.157695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fc35-4409-4080-baa7-a027646786ad None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:25.158548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2fc35-4409-4080-baa7-a027646786ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:25.159058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2fc35-4409-4080-baa7-a027646786ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:25.160612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2fc35-4409-4080-baa7-a027646786ad None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:25.175160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 358/697] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:25.177230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fc35-4409-4080-baa7-a027646786ad None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:25.207633 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d8dff-8918-4619-80fb-ebc7e108985f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:25.252709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286994a-1cf8-45c7-853f-95c820edc977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:25.253355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286994a-1cf8-45c7-853f-95c820edc977 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:25.253627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286994a-1cf8-45c7-853f-95c820edc977 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:25.253907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286994a-1cf8-45c7-853f-95c820edc977 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:25.255077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286994a-1cf8-45c7-853f-95c820edc977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:25.255201 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286994a-1cf8-45c7-853f-95c820edc977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:25.256531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286994a-1cf8-45c7-853f-95c820edc977 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:25.272512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 340/698] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:25.278014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286994a-1cf8-45c7-853f-95c820edc977 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:25.297844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:25.306711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fc35-4409-4080-baa7-a027646786ad None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:25.310063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:25.348003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72025e9f-9b49-4991-a25c-c1ddfcf3e478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:25.348910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72025e9f-9b49-4991-a25c-c1ddfcf3e478 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:25.349287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72025e9f-9b49-4991-a25c-c1ddfcf3e478 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:25.349710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72025e9f-9b49-4991-a25c-c1ddfcf3e478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:25.361832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72025e9f-9b49-4991-a25c-c1ddfcf3e478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:25.363424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72025e9f-9b49-4991-a25c-c1ddfcf3e478 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-72025e9f-9b49-4991-a25c-c1ddfcf3e478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:25.379342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:25.380425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:25.381335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:25.381335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:25.381335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:25.385604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 359/699] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:25.386866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f538c-6d47-45c7-abbd-82ab95d72eeb None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:25.387136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f538c-6d47-45c7-abbd-82ab95d72eeb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:25.387408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f538c-6d47-45c7-abbd-82ab95d72eeb None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:25.389226 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 360/700] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:25.390180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f538c-6d47-45c7-abbd-82ab95d72eeb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:25.406304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286994a-1cf8-45c7-853f-95c820edc977 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:25.442946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7e63f-5bfe-4e21-a89a-5ebc1935570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:25.443946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7e63f-5bfe-4e21-a89a-5ebc1935570e None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:25.444449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7e63f-5bfe-4e21-a89a-5ebc1935570e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:25.444942 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7e63f-5bfe-4e21-a89a-5ebc1935570e None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:25.445838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7e63f-5bfe-4e21-a89a-5ebc1935570e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:25.447497 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7e63f-5bfe-4e21-a89a-5ebc1935570e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:25.502687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f538c-6d47-45c7-abbd-82ab95d72eeb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:25.528552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 341/701] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:13:25 2022] POST /identity/v3/users => generated 254 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:13:25.534732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ace9-1a6b-4057-aff0-7574764f3b67 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:25.534732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ace9-1a6b-4057-aff0-7574764f3b67 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:25.534732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ace9-1a6b-4057-aff0-7574764f3b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:25.566092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1831ace9-1a6b-4057-aff0-7574764f3b67 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:25.567392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1831ace9-1a6b-4057-aff0-7574764f3b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:25.572647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3181-4f72-4bcc-be78-e82ac2ce34d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:25.573556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3181-4f72-4bcc-be78-e82ac2ce34d2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:25.573839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3181-4f72-4bcc-be78-e82ac2ce34d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:25.574109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3181-4f72-4bcc-be78-e82ac2ce34d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:25.574762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d3181-4f72-4bcc-be78-e82ac2ce34d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:25.575077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d3181-4f72-4bcc-be78-e82ac2ce34d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:25.576453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d3181-4f72-4bcc-be78-e82ac2ce34d2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:25.602425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 361/702] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:25.609915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3181-4f72-4bcc-be78-e82ac2ce34d2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:25.649044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:25.689927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:25.702254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 342/703] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:25.770896 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3181-4f72-4bcc-be78-e82ac2ce34d2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:25.807501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff6b27-e190-4ae7-9984-684e9beabdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:25.808134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff6b27-e190-4ae7-9984-684e9beabdfc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:25.808408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff6b27-e190-4ae7-9984-684e9beabdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:25.808684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff6b27-e190-4ae7-9984-684e9beabdfc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:25.817538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff6b27-e190-4ae7-9984-684e9beabdfc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:25.818971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff6b27-e190-4ae7-9984-684e9beabdfc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-48ff6b27-e190-4ae7-9984-684e9beabdfc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:25.832249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:25.833319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:25.834333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:25.834333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:25.834333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:25.834333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:25.834333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:25.834333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:25.834333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:25.834333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:25.834333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:25.834333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:25.841060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 362/704] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 1 05:13:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:25.842707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831ace9-1a6b-4057-aff0-7574764f3b67 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:25.956335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831ace9-1a6b-4057-aff0-7574764f3b67 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:25.994005 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3018349-20b1-4b31-b17b-c36a83475e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:25.994995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3018349-20b1-4b31-b17b-c36a83475e8c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:25.995505 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3018349-20b1-4b31-b17b-c36a83475e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:25.995985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3018349-20b1-4b31-b17b-c36a83475e8c None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:25.996908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3018349-20b1-4b31-b17b-c36a83475e8c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:25.997420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3018349-20b1-4b31-b17b-c36a83475e8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:25.999081 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3018349-20b1-4b31-b17b-c36a83475e8c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:26.010028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff6b27-e190-4ae7-9984-684e9beabdfc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:26.025034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 343/705] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 1 05:13:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:26.026600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3018349-20b1-4b31-b17b-c36a83475e8c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:26.095090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:26.137865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:26.146652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3018349-20b1-4b31-b17b-c36a83475e8c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:26.150448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff6b27-e190-4ae7-9984-684e9beabdfc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:26.186966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab6fb7-5f54-4c2a-8071-16c23f7703c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:26.187606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab6fb7-5f54-4c2a-8071-16c23f7703c0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:26.187951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab6fb7-5f54-4c2a-8071-16c23f7703c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:26.188446 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab6fb7-5f54-4c2a-8071-16c23f7703c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:26.196739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47983dc-b86e-4cf0-aa9f-f845169b1e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:26.198519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47983dc-b86e-4cf0-aa9f-f845169b1e1b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:26.199033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47983dc-b86e-4cf0-aa9f-f845169b1e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:26.199527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47983dc-b86e-4cf0-aa9f-f845169b1e1b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:26.200837 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47983dc-b86e-4cf0-aa9f-f845169b1e1b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:26.203950 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47983dc-b86e-4cf0-aa9f-f845169b1e1b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f47983dc-b86e-4cf0-aa9f-f845169b1e1b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:26.218432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:26.222505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:26.223571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 363/706] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Sep 1 05:13:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:26.226442 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b063adf-9ecb-4dbe-866a-5b8e745560f9 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:26.226884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b063adf-9ecb-4dbe-866a-5b8e745560f9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:26.227455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b063adf-9ecb-4dbe-866a-5b8e745560f9 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:26.229277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 364/707] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:26.235420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b063adf-9ecb-4dbe-866a-5b8e745560f9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:26.278270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab6fb7-5f54-4c2a-8071-16c23f7703c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:26.280022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab6fb7-5f54-4c2a-8071-16c23f7703c0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:26.336876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:26.375047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b063adf-9ecb-4dbe-866a-5b8e745560f9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:26.401442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 344/708] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:13:26.402951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8d970-f7ad-4700-b255-28a40a0eb348 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:26.403394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8d970-f7ad-4700-b255-28a40a0eb348 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:26.403815 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8d970-f7ad-4700-b255-28a40a0eb348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:26.426269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a1f8d970-f7ad-4700-b255-28a40a0eb348 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:26.426943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa89f37-96f3-428e-83c0-6409a1fb3169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:26.427695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa89f37-96f3-428e-83c0-6409a1fb3169 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:26.428080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a1f8d970-f7ad-4700-b255-28a40a0eb348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:26.428261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa89f37-96f3-428e-83c0-6409a1fb3169 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:26.428686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa89f37-96f3-428e-83c0-6409a1fb3169 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:26.429420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa89f37-96f3-428e-83c0-6409a1fb3169 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:26.430121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa89f37-96f3-428e-83c0-6409a1fb3169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:26.431959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa89f37-96f3-428e-83c0-6409a1fb3169 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:26.437241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 365/709] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 1 05:13:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:26.438829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa89f37-96f3-428e-83c0-6409a1fb3169 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:26.443631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:26.530129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 345/710] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:26.530129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8d970-f7ad-4700-b255-28a40a0eb348 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:26.576426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa89f37-96f3-428e-83c0-6409a1fb3169 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:26.650187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc274f43-de8a-4945-9c10-01d5ffd614b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:26.650842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc274f43-de8a-4945-9c10-01d5ffd614b3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:26.651132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc274f43-de8a-4945-9c10-01d5ffd614b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:26.651412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc274f43-de8a-4945-9c10-01d5ffd614b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:26.666178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:26.670482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc274f43-de8a-4945-9c10-01d5ffd614b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:26.671945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc274f43-de8a-4945-9c10-01d5ffd614b3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:26.691592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8d970-f7ad-4700-b255-28a40a0eb348 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-dc274f43-de8a-4945-9c10-01d5ffd614b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:26.714521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:26.716146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:26.718345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:26.718345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:26.718345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:26.725600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 366/711] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:26.729451 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc274f43-de8a-4945-9c10-01d5ffd614b3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:26.770086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2e805-682b-47a6-a10a-43974ea363d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:26.770733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2e805-682b-47a6-a10a-43974ea363d7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:26.771013 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2e805-682b-47a6-a10a-43974ea363d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:26.771277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2e805-682b-47a6-a10a-43974ea363d7 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:26.771907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2e805-682b-47a6-a10a-43974ea363d7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:26.772202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2e805-682b-47a6-a10a-43974ea363d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:26.773587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2e805-682b-47a6-a10a-43974ea363d7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:26.781499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 346/712] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 1 05:13:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:26.818968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:26.863123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc274f43-de8a-4945-9c10-01d5ffd614b3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:26.893790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2e805-682b-47a6-a10a-43974ea363d7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:26.919551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4077136-5553-4b06-a287-4c3dbdaa071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:26.920233 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4077136-5553-4b06-a287-4c3dbdaa071e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:26.920515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4077136-5553-4b06-a287-4c3dbdaa071e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:26.920788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4077136-5553-4b06-a287-4c3dbdaa071e None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:26.921450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4077136-5553-4b06-a287-4c3dbdaa071e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:26.921775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4077136-5553-4b06-a287-4c3dbdaa071e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:26.923750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4077136-5553-4b06-a287-4c3dbdaa071e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:26.953225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 367/713] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:26.970789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4077136-5553-4b06-a287-4c3dbdaa071e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:27.066940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2e805-682b-47a6-a10a-43974ea363d7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:27.118158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4077136-5553-4b06-a287-4c3dbdaa071e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:27.140819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1b4e0-a263-48ab-a869-5d94d09c4b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:27.142248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1b4e0-a263-48ab-a869-5d94d09c4b23 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:27.142586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1b4e0-a263-48ab-a869-5d94d09c4b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:27.142849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1b4e0-a263-48ab-a869-5d94d09c4b23 None admin] PATH_INFO: `/v3/services/network` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:27.143439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1b4e0-a263-48ab-a869-5d94d09c4b23 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:27.144820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1b4e0-a263-48ab-a869-5d94d09c4b23 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:27.154623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-2db1b4e0-a263-48ab-a869-5d94d09c4b23 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:27.167195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:27.170981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:27.171879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 347/714] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:26 2022] GET /identity/v3/services/network => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:27.178550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1b4e0-a263-48ab-a869-5d94d09c4b23 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:27.190847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6d93d-3640-4833-8e25-5b024d2b666d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:27.191762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6d93d-3640-4833-8e25-5b024d2b666d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:27.192243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6d93d-3640-4833-8e25-5b024d2b666d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:27.192679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6d93d-3640-4833-8e25-5b024d2b666d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:27.199449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:27.201401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6d93d-3640-4833-8e25-5b024d2b666d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:27.203223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6d93d-3640-4833-8e25-5b024d2b666d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c1e6d93d-3640-4833-8e25-5b024d2b666d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:27.217031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:27.219838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:27.220946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:27.220946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:27.220946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:27.220946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:27.220946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:27.220946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:27.220946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:27.220946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:27.220946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 368/715] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:27.229331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6d93d-3640-4833-8e25-5b024d2b666d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:27.346995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1b4e0-a263-48ab-a869-5d94d09c4b23 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:27.383329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da135ce9-566b-46cb-9659-771afae61f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:27.384297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da135ce9-566b-46cb-9659-771afae61f62 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:27.384792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da135ce9-566b-46cb-9659-771afae61f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:27.385266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da135ce9-566b-46cb-9659-771afae61f62 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:27.385958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da135ce9-566b-46cb-9659-771afae61f62 None admin] RBAC: Adding query filter params (name=network) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:27.386268 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da135ce9-566b-46cb-9659-771afae61f62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:27.387670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da135ce9-566b-46cb-9659-771afae61f62 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:27.393909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 348/716] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:13:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:27.395135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fb93f-4b60-4e38-b907-ca51632b056f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:27.395421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fb93f-4b60-4e38-b907-ca51632b056f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:27.396021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fb93f-4b60-4e38-b907-ca51632b056f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:27.396891 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 349/717] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:27.401858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fb93f-4b60-4e38-b907-ca51632b056f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:27.416458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6d93d-3640-4833-8e25-5b024d2b666d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:27.455255 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6271932-a602-48ad-af24-5c55de3aba46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:27.456205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6271932-a602-48ad-af24-5c55de3aba46 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:27.456638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6271932-a602-48ad-af24-5c55de3aba46 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:27.457060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6271932-a602-48ad-af24-5c55de3aba46 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:27.457935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6271932-a602-48ad-af24-5c55de3aba46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:27.458691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6271932-a602-48ad-af24-5c55de3aba46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:27.460302 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6271932-a602-48ad-af24-5c55de3aba46 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:27.476473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 369/718] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:27.477167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:27.479269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f07ce5-abec-458e-a783-36d6aafc84fb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:27.479706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f07ce5-abec-458e-a783-36d6aafc84fb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:27.480126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f07ce5-abec-458e-a783-36d6aafc84fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:27.499662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-01f07ce5-abec-458e-a783-36d6aafc84fb None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:27.501158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-01f07ce5-abec-458e-a783-36d6aafc84fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:27.509905 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fb93f-4b60-4e38-b907-ca51632b056f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:27.534308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:27.548751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ce4d-bb88-4c8f-b210-6c43eeea2dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:27.549410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ce4d-bb88-4c8f-b210-6c43eeea2dac None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:27.549724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ce4d-bb88-4c8f-b210-6c43eeea2dac None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:27.550000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ce4d-bb88-4c8f-b210-6c43eeea2dac None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:27.550648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ce4d-bb88-4c8f-b210-6c43eeea2dac None admin] RBAC: Adding query filter params (type=network) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:27.551091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ce4d-bb88-4c8f-b210-6c43eeea2dac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:27.552316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ce4d-bb88-4c8f-b210-6c43eeea2dac None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:27.557422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 350/719] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:13:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:27.558175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ce4d-bb88-4c8f-b210-6c43eeea2dac None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:27.580666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 370/720] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:27.682938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ce4d-bb88-4c8f-b210-6c43eeea2dac None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:27.727699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc539f-5757-427f-a521-b8cfa9c48bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:27.728341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc539f-5757-427f-a521-b8cfa9c48bea None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:27.728621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc539f-5757-427f-a521-b8cfa9c48bea None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:27.728944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc539f-5757-427f-a521-b8cfa9c48bea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:27.729689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc539f-5757-427f-a521-b8cfa9c48bea None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=7cda3c1f67e54ee08abc014399b07cc5) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:27.730012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc539f-5757-427f-a521-b8cfa9c48bea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:27.731521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc539f-5757-427f-a521-b8cfa9c48bea None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:27.738832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 351/721] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:27 2022] GET /identity/v3/role_assignments?user.id=7cda3c1f67e54ee08abc014399b07cc5&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:27.739442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:27.961674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f07ce5-abec-458e-a783-36d6aafc84fb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:28.076964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f07ce5-abec-458e-a783-36d6aafc84fb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:28.106646 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180c3a0-2f65-491d-ad38-02b5d680a9cd None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:28.106979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180c3a0-2f65-491d-ad38-02b5d680a9cd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:28.107205 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180c3a0-2f65-491d-ad38-02b5d680a9cd None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:28.109935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 352/722] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:28.115829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f5074-0360-485d-80e9-ebfe75b73551 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:28.116132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f5074-0360-485d-80e9-ebfe75b73551 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:28.116394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f5074-0360-485d-80e9-ebfe75b73551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:28.119819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2a06c-2ddc-45db-9ac8-c3da8e132866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:28.120668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2a06c-2ddc-45db-9ac8-c3da8e132866 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:28.121149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2a06c-2ddc-45db-9ac8-c3da8e132866 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:28.121641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2a06c-2ddc-45db-9ac8-c3da8e132866 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:28.133529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2a06c-2ddc-45db-9ac8-c3da8e132866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4af2a06c-2ddc-45db-9ac8-c3da8e132866 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:28.138848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:28.139794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:28.140655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:28.148204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 371/723] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:28.149849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2a06c-2ddc-45db-9ac8-c3da8e132866 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:28.154614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8a3f5074-0360-485d-80e9-ebfe75b73551 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:28.155844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8a3f5074-0360-485d-80e9-ebfe75b73551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:28.179608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:28.338290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2a06c-2ddc-45db-9ac8-c3da8e132866 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:28.340237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 353/724] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:13:28.374070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283076b-2114-4226-8718-61634cfee3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:28.374686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283076b-2114-4226-8718-61634cfee3fc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:28.374967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283076b-2114-4226-8718-61634cfee3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:28.375274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283076b-2114-4226-8718-61634cfee3fc None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:28.375967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283076b-2114-4226-8718-61634cfee3fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:28.376256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283076b-2114-4226-8718-61634cfee3fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:28.377631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283076b-2114-4226-8718-61634cfee3fc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:28.386404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 372/725] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:28.392742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f5074-0360-485d-80e9-ebfe75b73551 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:28.601158 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f5074-0360-485d-80e9-ebfe75b73551 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:28.644909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:28.650448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af55ce-902c-4246-8bcf-fd79ee1081a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:28.651074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af55ce-902c-4246-8bcf-fd79ee1081a3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:28.651344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af55ce-902c-4246-8bcf-fd79ee1081a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:28.651608 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af55ce-902c-4246-8bcf-fd79ee1081a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:28.658674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283076b-2114-4226-8718-61634cfee3fc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:28.667628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af55ce-902c-4246-8bcf-fd79ee1081a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:28.669071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af55ce-902c-4246-8bcf-fd79ee1081a3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-05af55ce-902c-4246-8bcf-fd79ee1081a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:28.685082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:28.686112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:28.687207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:28.687207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:28.687207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:28.687207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 354/726] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:28.693775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af55ce-902c-4246-8bcf-fd79ee1081a3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:28.846200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:28.868375 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283076b-2114-4226-8718-61634cfee3fc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:28.889571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af55ce-902c-4246-8bcf-fd79ee1081a3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:28.942899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326937ef-d3cf-434e-a14b-30f9338b79b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:28.943507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326937ef-d3cf-434e-a14b-30f9338b79b2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:28.944297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326937ef-d3cf-434e-a14b-30f9338b79b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:28.944567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326937ef-d3cf-434e-a14b-30f9338b79b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:28.945215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326937ef-d3cf-434e-a14b-30f9338b79b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:28.945540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326937ef-d3cf-434e-a14b-30f9338b79b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:28.949010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afa847-4f82-4047-a959-5699422450cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:28.949663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afa847-4f82-4047-a959-5699422450cd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:28.949940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afa847-4f82-4047-a959-5699422450cd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:28.950200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afa847-4f82-4047-a959-5699422450cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:28.950964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326937ef-d3cf-434e-a14b-30f9338b79b2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:28.971317 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19afa847-4f82-4047-a959-5699422450cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:28.979251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 355/727] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-19afa847-4f82-4047-a959-5699422450cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:28.979684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:28.981645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:28.982536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 373/728] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:28.993944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326937ef-d3cf-434e-a14b-30f9338b79b2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:28.995224 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afa847-4f82-4047-a959-5699422450cd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:29.082662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:29.184791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326937ef-d3cf-434e-a14b-30f9338b79b2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:29.195144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afa847-4f82-4047-a959-5699422450cd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:29.243427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896916d5-fb12-4590-8727-51b05be80be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:29.244278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896916d5-fb12-4590-8727-51b05be80be8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:29.244553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896916d5-fb12-4590-8727-51b05be80be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:29.245219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896916d5-fb12-4590-8727-51b05be80be8 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:29.246273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896916d5-fb12-4590-8727-51b05be80be8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:29.246573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896916d5-fb12-4590-8727-51b05be80be8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:29.247996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896916d5-fb12-4590-8727-51b05be80be8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:29.249865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00826722-fd44-40c2-be8d-b0c2789248af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:29.250963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00826722-fd44-40c2-be8d-b0c2789248af None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:29.251243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00826722-fd44-40c2-be8d-b0c2789248af None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:29.251519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00826722-fd44-40c2-be8d-b0c2789248af None admin] PATH_INFO: `/v3/users/swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:29.258123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 356/729] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:29.259398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e5639-a74e-49c0-be3a-fe575d5c4fdb None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:29.259835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e5639-a74e-49c0-be3a-fe575d5c4fdb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:29.260242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e5639-a74e-49c0-be3a-fe575d5c4fdb None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:29.260653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00826722-fd44-40c2-be8d-b0c2789248af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:29.262144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00826722-fd44-40c2-be8d-b0c2789248af None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:29.266436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 357/730] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:29.268247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e5639-a74e-49c0-be3a-fe575d5c4fdb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-00826722-fd44-40c2-be8d-b0c2789248af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:29.278654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:29.280688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:29.282711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 374/731] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:29.283466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4231a-89ac-4570-b696-3f4df6dc58db None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:29.284484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4231a-89ac-4570-b696-3f4df6dc58db None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:29.284484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4231a-89ac-4570-b696-3f4df6dc58db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:29.309642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bdd4231a-89ac-4570-b696-3f4df6dc58db None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:29.311954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bdd4231a-89ac-4570-b696-3f4df6dc58db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:29.314346 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:29.408639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 375/732] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:29.408639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4231a-89ac-4570-b696-3f4df6dc58db None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:29.409500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e5639-a74e-49c0-be3a-fe575d5c4fdb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:29.417590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:29.492064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff7dca-2575-4079-8ca4-05a5e28243b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:29.492619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff7dca-2575-4079-8ca4-05a5e28243b9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:29.492900 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff7dca-2575-4079-8ca4-05a5e28243b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:29.493175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff7dca-2575-4079-8ca4-05a5e28243b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:29.522323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff7dca-2575-4079-8ca4-05a5e28243b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:29.524167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff7dca-2575-4079-8ca4-05a5e28243b9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-ebff7dca-2575-4079-8ca4-05a5e28243b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:29.554943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:29.555948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:29.556879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:29.556879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:29.556879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:29.558724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 358/733] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:29.565004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff7dca-2575-4079-8ca4-05a5e28243b9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:29.586537 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4231a-89ac-4570-b696-3f4df6dc58db None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:29.643591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fe7eb-5c2f-49e4-854d-8793232bfa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:29.644213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fe7eb-5c2f-49e4-854d-8793232bfa5c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:29.644497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fe7eb-5c2f-49e4-854d-8793232bfa5c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:29.644771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fe7eb-5c2f-49e4-854d-8793232bfa5c None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:29.645805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fe7eb-5c2f-49e4-854d-8793232bfa5c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:29.646110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fe7eb-5c2f-49e4-854d-8793232bfa5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:29.648312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fe7eb-5c2f-49e4-854d-8793232bfa5c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:29.665782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:29.679425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 376/734] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 1 05:13:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:29.681779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fe7eb-5c2f-49e4-854d-8793232bfa5c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:29.735591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff7dca-2575-4079-8ca4-05a5e28243b9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:29.777107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd6525-55b4-4d22-9ef8-b7ca002a10d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:29.777988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd6525-55b4-4d22-9ef8-b7ca002a10d2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:29.778417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd6525-55b4-4d22-9ef8-b7ca002a10d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:29.779426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd6525-55b4-4d22-9ef8-b7ca002a10d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:29.780237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd6525-55b4-4d22-9ef8-b7ca002a10d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:29.780694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd6525-55b4-4d22-9ef8-b7ca002a10d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:29.783661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd6525-55b4-4d22-9ef8-b7ca002a10d2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:29.800935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 359/735] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:29.802344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd6525-55b4-4d22-9ef8-b7ca002a10d2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:29.826332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:29.861625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fe7eb-5c2f-49e4-854d-8793232bfa5c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:29.907416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435f405-85bb-45f6-b1c6-3069fed17977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:29.908460 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435f405-85bb-45f6-b1c6-3069fed17977 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:29.908899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435f405-85bb-45f6-b1c6-3069fed17977 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:29.909182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435f405-85bb-45f6-b1c6-3069fed17977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:29.935348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd6525-55b4-4d22-9ef8-b7ca002a10d2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:29.972790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee87b2e-8672-4ac1-9a07-51e90c902a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:29.973394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee87b2e-8672-4ac1-9a07-51e90c902a86 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:29.973705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee87b2e-8672-4ac1-9a07-51e90c902a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:29.973977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee87b2e-8672-4ac1-9a07-51e90c902a86 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:29.974625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee87b2e-8672-4ac1-9a07-51e90c902a86 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:29.975998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee87b2e-8672-4ac1-9a07-51e90c902a86 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:29.982993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:29.990106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435f405-85bb-45f6-b1c6-3069fed17977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:29.991536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435f405-85bb-45f6-b1c6-3069fed17977 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:30.055265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 360/736] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:13:29 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:30.056135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee87b2e-8672-4ac1-9a07-51e90c902a86 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:30.100352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 377/737] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:30.102094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b06bc-3cd9-4020-a04e-8f2f344b30d4 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.102481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b06bc-3cd9-4020-a04e-8f2f344b30d4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.102762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b06bc-3cd9-4020-a04e-8f2f344b30d4 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.104232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 378/738] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:30.105116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f60b8-e41d-406f-940a-a428cdd1bbc5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.105391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f60b8-e41d-406f-940a-a428cdd1bbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.105694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f60b8-e41d-406f-940a-a428cdd1bbc5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.107022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 379/739] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:30.112763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f60b8-e41d-406f-940a-a428cdd1bbc5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:30.177380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:30.214106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee87b2e-8672-4ac1-9a07-51e90c902a86 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:30.226175 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:30.232980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f60b8-e41d-406f-940a-a428cdd1bbc5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:30.255731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237426e-63b3-4cea-8e5e-b3a671f4940a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:30.256401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237426e-63b3-4cea-8e5e-b3a671f4940a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.256641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237426e-63b3-4cea-8e5e-b3a671f4940a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.256916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237426e-63b3-4cea-8e5e-b3a671f4940a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.265118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237426e-63b3-4cea-8e5e-b3a671f4940a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:30.267343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237426e-63b3-4cea-8e5e-b3a671f4940a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:30.271904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de2e85-f0b4-4fcf-a9e9-033a52f62362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:30.272746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de2e85-f0b4-4fcf-a9e9-033a52f62362 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.273197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de2e85-f0b4-4fcf-a9e9-033a52f62362 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.273690 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de2e85-f0b4-4fcf-a9e9-033a52f62362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-3237426e-63b3-4cea-8e5e-b3a671f4940a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:30.279417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:30.280351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:30.281901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 361/740] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 1 05:13:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:30.282918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05f06d-4327-409f-a460-0def75d46a86 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.282918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05f06d-4327-409f-a460-0def75d46a86 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.283582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05f06d-4327-409f-a460-0def75d46a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.290888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de2e85-f0b4-4fcf-a9e9-033a52f62362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:30.292416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de2e85-f0b4-4fcf-a9e9-033a52f62362 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:30.301900 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4c05f06d-4327-409f-a460-0def75d46a86 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:30.303092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4c05f06d-4327-409f-a460-0def75d46a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-34de2e85-f0b4-4fcf-a9e9-033a52f62362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:30.309276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:30.310264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:30.311887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:30.311887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:30.311887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:30.311887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 380/741] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:30.312172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203e086-672e-4f8f-aaa6-fcafbf805cb7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.312254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203e086-672e-4f8f-aaa6-fcafbf805cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.312499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203e086-672e-4f8f-aaa6-fcafbf805cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.334311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8203e086-672e-4f8f-aaa6-fcafbf805cb7 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:30.335500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8203e086-672e-4f8f-aaa6-fcafbf805cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:30.379447 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 362/742] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:30.380299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05f06d-4327-409f-a460-0def75d46a86 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:30.417955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 381/743] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:13:30.419224 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203e086-672e-4f8f-aaa6-fcafbf805cb7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:30.487624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:30.561955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:30.571641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05f06d-4327-409f-a460-0def75d46a86 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:30.574749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203e086-672e-4f8f-aaa6-fcafbf805cb7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:30.611152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcc0f0-1797-44b5-8e4a-6086507a5ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:30.611762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcc0f0-1797-44b5-8e4a-6086507a5ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.612038 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcc0f0-1797-44b5-8e4a-6086507a5ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.612306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcc0f0-1797-44b5-8e4a-6086507a5ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.612954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcc0f0-1797-44b5-8e4a-6086507a5ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:30.613255 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcc0f0-1797-44b5-8e4a-6086507a5ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:30.614644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcc0f0-1797-44b5-8e4a-6086507a5ec8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:30.626695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 382/744] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:30.632070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcc0f0-1797-44b5-8e4a-6086507a5ec8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:30.651710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2fb88-1d62-4361-a778-48c634ca391e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:30.652334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2fb88-1d62-4361-a778-48c634ca391e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.652672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2fb88-1d62-4361-a778-48c634ca391e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.652954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2fb88-1d62-4361-a778-48c634ca391e None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.653774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2fb88-1d62-4361-a778-48c634ca391e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:30.654087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2fb88-1d62-4361-a778-48c634ca391e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:30.655589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2fb88-1d62-4361-a778-48c634ca391e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:30.692221 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 363/745] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 1 05:13:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:30.698329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2fb88-1d62-4361-a778-48c634ca391e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:30.776671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcc0f0-1797-44b5-8e4a-6086507a5ec8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:30.814699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90335e22-b5dd-4dc2-962c-c9f4a3d78a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:30.815247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90335e22-b5dd-4dc2-962c-c9f4a3d78a45 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.815521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90335e22-b5dd-4dc2-962c-c9f4a3d78a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.815789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90335e22-b5dd-4dc2-962c-c9f4a3d78a45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.823005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90335e22-b5dd-4dc2-962c-c9f4a3d78a45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:30.824670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90335e22-b5dd-4dc2-962c-c9f4a3d78a45 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:30.826313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2fb88-1d62-4361-a778-48c634ca391e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:30.826618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-90335e22-b5dd-4dc2-962c-c9f4a3d78a45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:30.836724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:30.837676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:30.838632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:30.838632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:30.838632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:30.838632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:30.838632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:30.838632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:30.838632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:30.838632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:30.838632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 383/746] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:30.838632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90335e22-b5dd-4dc2-962c-c9f4a3d78a45 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:30.853117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:30.867573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90464fac-5ff9-4e8c-a343-c8497042da1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:30.868169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90464fac-5ff9-4e8c-a343-c8497042da1b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.868440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90464fac-5ff9-4e8c-a343-c8497042da1b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.868708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90464fac-5ff9-4e8c-a343-c8497042da1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.942050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90464fac-5ff9-4e8c-a343-c8497042da1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:30.943670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90464fac-5ff9-4e8c-a343-c8497042da1b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:30.948495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90335e22-b5dd-4dc2-962c-c9f4a3d78a45 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:30.984068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121faeb3-d35c-4f3b-a2c8-2140ec45d94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:30.984978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121faeb3-d35c-4f3b-a2c8-2140ec45d94e None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:30.985422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121faeb3-d35c-4f3b-a2c8-2140ec45d94e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:30.985871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121faeb3-d35c-4f3b-a2c8-2140ec45d94e None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:30.986669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121faeb3-d35c-4f3b-a2c8-2140ec45d94e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:30.988165 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121faeb3-d35c-4f3b-a2c8-2140ec45d94e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:31.052374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 364/747] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:13:31.052374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90464fac-5ff9-4e8c-a343-c8497042da1b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:31.058933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 384/748] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:13:30 2022] POST /identity/v3/services => generated 240 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:31.066623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121faeb3-d35c-4f3b-a2c8-2140ec45d94e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:31.081659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:31.141949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:31.200050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90464fac-5ff9-4e8c-a343-c8497042da1b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:31.216052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121faeb3-d35c-4f3b-a2c8-2140ec45d94e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:31.249230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaed22d-2d6e-401c-b4b6-3a6923a1d317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:31.249667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaed22d-2d6e-401c-b4b6-3a6923a1d317 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:31.249981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaed22d-2d6e-401c-b4b6-3a6923a1d317 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:31.250255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaed22d-2d6e-401c-b4b6-3a6923a1d317 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:31.265036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaed22d-2d6e-401c-b4b6-3a6923a1d317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:31.269916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080d8b2-8d94-4662-8948-82abaf27404c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-9aaed22d-2d6e-401c-b4b6-3a6923a1d317 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:31.270423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:31.271551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:31.272656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:31.273439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080d8b2-8d94-4662-8948-82abaf27404c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:31.273439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080d8b2-8d94-4662-8948-82abaf27404c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:31.273439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080d8b2-8d94-4662-8948-82abaf27404c None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:31.273439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080d8b2-8d94-4662-8948-82abaf27404c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:31.273439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080d8b2-8d94-4662-8948-82abaf27404c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:31.273439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080d8b2-8d94-4662-8948-82abaf27404c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:31.279484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 365/749] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:31.279484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaed22d-2d6e-401c-b4b6-3a6923a1d317 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:31.286898 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 385/750] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:31.287632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080d8b2-8d94-4662-8948-82abaf27404c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:31.381528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:31.396052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080d8b2-8d94-4662-8948-82abaf27404c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:31.405496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaed22d-2d6e-401c-b4b6-3a6923a1d317 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:31.434787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a4f89-e2e5-4a8d-9226-752208ca4d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:31.435779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a4f89-e2e5-4a8d-9226-752208ca4d34 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:31.436347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a4f89-e2e5-4a8d-9226-752208ca4d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:31.436947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a4f89-e2e5-4a8d-9226-752208ca4d34 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:31.437896 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1a4f89-e2e5-4a8d-9226-752208ca4d34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:31.438441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1a4f89-e2e5-4a8d-9226-752208ca4d34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:31.440209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1a4f89-e2e5-4a8d-9226-752208ca4d34 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:31.449358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1083ad2-4427-4bcb-aebb-f5a41e148a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:31.450037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1083ad2-4427-4bcb-aebb-f5a41e148a57 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:31.450317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1083ad2-4427-4bcb-aebb-f5a41e148a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:31.450592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1083ad2-4427-4bcb-aebb-f5a41e148a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:31.456760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:31.457088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 386/751] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:31.458350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a4f89-e2e5-4a8d-9226-752208ca4d34 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:31.462076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1083ad2-4427-4bcb-aebb-f5a41e148a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:31.464543 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1083ad2-4427-4bcb-aebb-f5a41e148a57 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-b1083ad2-4427-4bcb-aebb-f5a41e148a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:31.487848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:31.489304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:31.490270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:31.490270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:31.490270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:31.493906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 366/752] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:31.495027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1083ad2-4427-4bcb-aebb-f5a41e148a57 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:31.580607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a4f89-e2e5-4a8d-9226-752208ca4d34 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:31.621792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de414f98-e407-4dca-8be6-3059247786d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:31.622846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de414f98-e407-4dca-8be6-3059247786d4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:31.623336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de414f98-e407-4dca-8be6-3059247786d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:31.623950 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de414f98-e407-4dca-8be6-3059247786d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:31.625042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de414f98-e407-4dca-8be6-3059247786d4 None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=b723681faf15434e8529e8fc56c6e9b8) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:31.625607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de414f98-e407-4dca-8be6-3059247786d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:31.627570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de414f98-e407-4dca-8be6-3059247786d4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:31.630690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1083ad2-4427-4bcb-aebb-f5a41e148a57 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:31.634632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 387/753] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:31 2022] GET /identity/v3/role_assignments?user.id=b723681faf15434e8529e8fc56c6e9b8&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:31.635701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de414f98-e407-4dca-8be6-3059247786d4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:31.696466 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:31.708403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5c195-f247-470f-b0db-edd93f44f56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:31.711524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c195-f247-470f-b0db-edd93f44f56a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:31.712039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c195-f247-470f-b0db-edd93f44f56a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:31.712471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c195-f247-470f-b0db-edd93f44f56a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:31.733503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:31.738353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5c195-f247-470f-b0db-edd93f44f56a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:31.739820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5c195-f247-470f-b0db-edd93f44f56a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-c4d5c195-f247-470f-b0db-edd93f44f56a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:31.774517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:31.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:31.776465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:31.776465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:31.776465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:31.777092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 367/754] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:31.782737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5c195-f247-470f-b0db-edd93f44f56a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:31.857101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de414f98-e407-4dca-8be6-3059247786d4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:31.911765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5c195-f247-470f-b0db-edd93f44f56a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:31.951186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995afa3c-c343-4ac4-887b-d1db2b2b0e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:31.951186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995afa3c-c343-4ac4-887b-d1db2b2b0e36 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:31.951186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995afa3c-c343-4ac4-887b-d1db2b2b0e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:31.951186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995afa3c-c343-4ac4-887b-d1db2b2b0e36 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:31.951601 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995afa3c-c343-4ac4-887b-d1db2b2b0e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:31.951601 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995afa3c-c343-4ac4-887b-d1db2b2b0e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:31.951601 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995afa3c-c343-4ac4-887b-d1db2b2b0e36 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:31.958018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00451bb-e3ed-4c09-b995-ad12cd1f9a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:31.958858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00451bb-e3ed-4c09-b995-ad12cd1f9a5f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:31.959332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00451bb-e3ed-4c09-b995-ad12cd1f9a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:31.959813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00451bb-e3ed-4c09-b995-ad12cd1f9a5f None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:31.960641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00451bb-e3ed-4c09-b995-ad12cd1f9a5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:31.961137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00451bb-e3ed-4c09-b995-ad12cd1f9a5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:31.962710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00451bb-e3ed-4c09-b995-ad12cd1f9a5f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:31.979139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 388/755] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:31.985153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995afa3c-c343-4ac4-887b-d1db2b2b0e36 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:32.000501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 368/756] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:32.006252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00451bb-e3ed-4c09-b995-ad12cd1f9a5f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:32.088592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:32.137821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:32.141476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995afa3c-c343-4ac4-887b-d1db2b2b0e36 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:32.183427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00451bb-e3ed-4c09-b995-ad12cd1f9a5f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:32.194303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20a3b2-1257-4d23-bdf1-35114bd8e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:32.194952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20a3b2-1257-4d23-bdf1-35114bd8e959 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:32.195232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20a3b2-1257-4d23-bdf1-35114bd8e959 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:32.195503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20a3b2-1257-4d23-bdf1-35114bd8e959 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:32.203661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20a3b2-1257-4d23-bdf1-35114bd8e959 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:32.205264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20a3b2-1257-4d23-bdf1-35114bd8e959 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0f20a3b2-1257-4d23-bdf1-35114bd8e959 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:32.217062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:32.218061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:32.219582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:32.219582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:32.219582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:32.219582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:32.219582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:32.219582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:32.219582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:32.219582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:32.219582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 389/757] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:32.220814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6189c-9b67-44fa-b5d8-8668d08587dd None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:32.220981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6189c-9b67-44fa-b5d8-8668d08587dd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:32.221171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6189c-9b67-44fa-b5d8-8668d08587dd None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:32.222736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 390/758] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:32.224953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6189c-9b67-44fa-b5d8-8668d08587dd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:32.230167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a385ae-8a8d-4a97-a846-bac9a53d9e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:32.230815 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a385ae-8a8d-4a97-a846-bac9a53d9e3c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:32.231096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a385ae-8a8d-4a97-a846-bac9a53d9e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:32.231370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a385ae-8a8d-4a97-a846-bac9a53d9e3c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:32.240741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a385ae-8a8d-4a97-a846-bac9a53d9e3c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:32.242260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a385ae-8a8d-4a97-a846-bac9a53d9e3c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d4a385ae-8a8d-4a97-a846-bac9a53d9e3c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:32.255326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:32.257626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:32.259853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:32.259853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:32.259853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:32.259853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:32.259853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:32.259853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:32.259853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:32.259853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:32.259853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:32.259853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:32.265688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 369/759] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 1 05:13:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:32.265688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b38ba-ae5a-4ce5-b43a-a2e2df224449 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:32.265688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b38ba-ae5a-4ce5-b43a-a2e2df224449 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:32.265688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b38ba-ae5a-4ce5-b43a-a2e2df224449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:32.282798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-980b38ba-ae5a-4ce5-b43a-a2e2df224449 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:32.284000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-980b38ba-ae5a-4ce5-b43a-a2e2df224449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:32.353388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6189c-9b67-44fa-b5d8-8668d08587dd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:32.372594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 370/760] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:32.373697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b38ba-ae5a-4ce5-b43a-a2e2df224449 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:32.422742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64a472-7f2f-4f25-82b0-7a3b26733600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:32.423371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64a472-7f2f-4f25-82b0-7a3b26733600 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:32.423644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64a472-7f2f-4f25-82b0-7a3b26733600 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:32.423912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64a472-7f2f-4f25-82b0-7a3b26733600 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:32.424603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64a472-7f2f-4f25-82b0-7a3b26733600 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:32.424904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64a472-7f2f-4f25-82b0-7a3b26733600 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:32.426343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64a472-7f2f-4f25-82b0-7a3b26733600 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:32.445342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:32.448394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 391/761] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:32.454636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64a472-7f2f-4f25-82b0-7a3b26733600 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:32.477863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:32.577366 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64a472-7f2f-4f25-82b0-7a3b26733600 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:32.615027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b38ba-ae5a-4ce5-b43a-a2e2df224449 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:32.635920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9878b-b804-4f90-8297-a42a06b07383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:32.636545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9878b-b804-4f90-8297-a42a06b07383 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:32.636820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9878b-b804-4f90-8297-a42a06b07383 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:32.637092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9878b-b804-4f90-8297-a42a06b07383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:32.637853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9878b-b804-4f90-8297-a42a06b07383 None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=f58aa170e3004197a15dc2dc868d8681) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:32.638162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9878b-b804-4f90-8297-a42a06b07383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:32.639600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9878b-b804-4f90-8297-a42a06b07383 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:32.654772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 392/762] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:32 2022] GET /identity/v3/role_assignments?user.id=f58aa170e3004197a15dc2dc868d8681&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:32.690077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec4b97-36b0-42cb-933b-4c83c97f0a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:32.690721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec4b97-36b0-42cb-933b-4c83c97f0a8b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:32.690999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec4b97-36b0-42cb-933b-4c83c97f0a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:32.691273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec4b97-36b0-42cb-933b-4c83c97f0a8b None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:32.692029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec4b97-36b0-42cb-933b-4c83c97f0a8b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:32.692337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec4b97-36b0-42cb-933b-4c83c97f0a8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:32.693855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec4b97-36b0-42cb-933b-4c83c97f0a8b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:32.807420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9878b-b804-4f90-8297-a42a06b07383 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:32.858429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 371/763] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 1 05:13:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:32.866489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec4b97-36b0-42cb-933b-4c83c97f0a8b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:32.976695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:32.995427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9878b-b804-4f90-8297-a42a06b07383 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:33.041967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:33.048979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec4b97-36b0-42cb-933b-4c83c97f0a8b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:33.056424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d929827-bf3c-4af0-bb17-109c2e65c4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:33.056424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d929827-bf3c-4af0-bb17-109c2e65c4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:33.056424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d929827-bf3c-4af0-bb17-109c2e65c4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:33.056424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d929827-bf3c-4af0-bb17-109c2e65c4c4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:33.057004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d929827-bf3c-4af0-bb17-109c2e65c4c4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:33.057004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d929827-bf3c-4af0-bb17-109c2e65c4c4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7d929827-bf3c-4af0-bb17-109c2e65c4c4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:33.068772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:33.069830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:33.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 393/764] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Sep 1 05:13:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:33.081242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d929827-bf3c-4af0-bb17-109c2e65c4c4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:33.103374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25887e66-88eb-484c-9872-52a473639990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:33.104665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25887e66-88eb-484c-9872-52a473639990 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:33.105381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25887e66-88eb-484c-9872-52a473639990 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:33.105857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25887e66-88eb-484c-9872-52a473639990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:33.119612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25887e66-88eb-484c-9872-52a473639990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:33.122109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25887e66-88eb-484c-9872-52a473639990 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-25887e66-88eb-484c-9872-52a473639990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:33.144873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:33.145954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:33.146845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:33.146845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:33.146845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:33.148921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 372/765] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:33.157913 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25887e66-88eb-484c-9872-52a473639990 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:33.298313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25887e66-88eb-484c-9872-52a473639990 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:33.337335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fe52a-3c94-489c-9030-769a992fd2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:33.337990 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fe52a-3c94-489c-9030-769a992fd2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:33.338270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fe52a-3c94-489c-9030-769a992fd2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:33.338548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fe52a-3c94-489c-9030-769a992fd2f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:33.339206 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fe52a-3c94-489c-9030-769a992fd2f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:33.339322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d929827-bf3c-4af0-bb17-109c2e65c4c4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:33.339510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fe52a-3c94-489c-9030-769a992fd2f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:33.340971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fe52a-3c94-489c-9030-769a992fd2f9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:33.357262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 373/766] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:33.357262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9399c3-e668-4077-bc06-7bdabbe91ed4 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:33.357262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9399c3-e668-4077-bc06-7bdabbe91ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:33.357262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9399c3-e668-4077-bc06-7bdabbe91ed4 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:33.361027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 374/767] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:33.363438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bdf12-95ad-4b35-a135-d202e03e381d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:33.363728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bdf12-95ad-4b35-a135-d202e03e381d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:33.364008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bdf12-95ad-4b35-a135-d202e03e381d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:33.371945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:33.385056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7c7bdf12-95ad-4b35-a135-d202e03e381d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:33.386365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7c7bdf12-95ad-4b35-a135-d202e03e381d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:33.399862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189127a7-144c-41f9-96db-d3ede1640b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:33.400941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189127a7-144c-41f9-96db-d3ede1640b50 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:33.401404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189127a7-144c-41f9-96db-d3ede1640b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:33.401887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189127a7-144c-41f9-96db-d3ede1640b50 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:33.402734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189127a7-144c-41f9-96db-d3ede1640b50 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:33.403277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189127a7-144c-41f9-96db-d3ede1640b50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:33.404860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189127a7-144c-41f9-96db-d3ede1640b50 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:33.416140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 394/768] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 1 05:13:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:33.417479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189127a7-144c-41f9-96db-d3ede1640b50 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:33.435419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:33.469610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 375/769] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:13:33.470523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bdf12-95ad-4b35-a135-d202e03e381d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:33.532051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189127a7-144c-41f9-96db-d3ede1640b50 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:33.569047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebf13e-d89b-4e16-91e4-cf37c61a2176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:33.570025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebf13e-d89b-4e16-91e4-cf37c61a2176 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:33.570483 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebf13e-d89b-4e16-91e4-cf37c61a2176 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:33.570928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebf13e-d89b-4e16-91e4-cf37c61a2176 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:33.578504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebf13e-d89b-4e16-91e4-cf37c61a2176 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:33.580204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebf13e-d89b-4e16-91e4-cf37c61a2176 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-aaebf13e-d89b-4e16-91e4-cf37c61a2176 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:33.591933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:33.592862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:33.594386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:33.594386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:33.594386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:33.594386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:33.594386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:33.594386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:33.594386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:33.594386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:33.596372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 395/770] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:33.602130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebf13e-d89b-4e16-91e4-cf37c61a2176 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:33.643005 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bdf12-95ad-4b35-a135-d202e03e381d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:33.678834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:33.705721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770969c3-8567-47df-b77e-29141ea1f925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:33.706423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770969c3-8567-47df-b77e-29141ea1f925 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:33.706703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770969c3-8567-47df-b77e-29141ea1f925 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:33.706978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770969c3-8567-47df-b77e-29141ea1f925 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:33.707781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770969c3-8567-47df-b77e-29141ea1f925 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:33.707909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770969c3-8567-47df-b77e-29141ea1f925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:33.709377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770969c3-8567-47df-b77e-29141ea1f925 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:33.710832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:33.714634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebf13e-d89b-4e16-91e4-cf37c61a2176 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:33.724010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 376/771] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 1 05:13:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:33.735863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770969c3-8567-47df-b77e-29141ea1f925 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:33.761839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db80345-c838-4c8a-bad1-3d778f5480c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:33.762503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db80345-c838-4c8a-bad1-3d778f5480c3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:33.762827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db80345-c838-4c8a-bad1-3d778f5480c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:33.763100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db80345-c838-4c8a-bad1-3d778f5480c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:33.763850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db80345-c838-4c8a-bad1-3d778f5480c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:33.764154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db80345-c838-4c8a-bad1-3d778f5480c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:33.765739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db80345-c838-4c8a-bad1-3d778f5480c3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:33.780916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 396/772] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:33.787525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db80345-c838-4c8a-bad1-3d778f5480c3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:33.897349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db80345-c838-4c8a-bad1-3d778f5480c3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:33.931344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:33.940181 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ecef1-513c-462f-87c2-e8da9e81e3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:33.940858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ecef1-513c-462f-87c2-e8da9e81e3ea None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:33.941135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ecef1-513c-462f-87c2-e8da9e81e3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:33.941420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ecef1-513c-462f-87c2-e8da9e81e3ea None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/7cda3c1f67e54ee08abc014399b07cc5/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:33.943610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ecef1-513c-462f-87c2-e8da9e81e3ea None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=7cda3c1f67e54ee08abc014399b07cc5, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:33.952825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ecef1-513c-462f-87c2-e8da9e81e3ea None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:33.964815 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770969c3-8567-47df-b77e-29141ea1f925 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:34.002909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710b162-abd0-4d09-a643-1ee863938ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:34.003647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710b162-abd0-4d09-a643-1ee863938ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.003852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710b162-abd0-4d09-a643-1ee863938ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.004146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710b162-abd0-4d09-a643-1ee863938ce3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.004838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710b162-abd0-4d09-a643-1ee863938ce3 None admin] RBAC: Adding query filter params (interface=public, service_id=e8f62c75755641ba83524bc6bd131add, region_id=RegionOne) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:34.005430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710b162-abd0-4d09-a643-1ee863938ce3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:34.006603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710b162-abd0-4d09-a643-1ee863938ce3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:34.013590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 397/773] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:13:33 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/7cda3c1f67e54ee08abc014399b07cc5/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:13:34.019588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 377/774] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 1 05:13:33 2022] GET /identity/v3/endpoints?service_id=e8f62c75755641ba83524bc6bd131add&interface=public®ion_id=RegionOne => generated 194 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:34.019588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710b162-abd0-4d09-a643-1ee863938ce3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:34.022719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ecef1-513c-462f-87c2-e8da9e81e3ea None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:34.111139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:34.173612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710b162-abd0-4d09-a643-1ee863938ce3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:34.176301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ecef1-513c-462f-87c2-e8da9e81e3ea None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:34.224963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb6763-6b8f-4528-b72e-b4f2c36e6656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:34.228145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb6763-6b8f-4528-b72e-b4f2c36e6656 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.228145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb6763-6b8f-4528-b72e-b4f2c36e6656 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.228145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb6763-6b8f-4528-b72e-b4f2c36e6656 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.228145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb6763-6b8f-4528-b72e-b4f2c36e6656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:34.228669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb6763-6b8f-4528-b72e-b4f2c36e6656 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:34.229985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800ba28-f527-4642-88e5-d9296e23b555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:34.230953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800ba28-f527-4642-88e5-d9296e23b555 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.231239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800ba28-f527-4642-88e5-d9296e23b555 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.231512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800ba28-f527-4642-88e5-d9296e23b555 None admin] PATH_INFO: `/v3/services/network` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.232278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800ba28-f527-4642-88e5-d9296e23b555 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:34.233662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800ba28-f527-4642-88e5-d9296e23b555 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:34.237368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 398/775] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 1 05:13:34 2022] GET /identity/v3/services => generated 772 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:34.237368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd08016-1dca-4473-893c-e4a936038c49 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.237368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd08016-1dca-4473-893c-e4a936038c49 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.237368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd08016-1dca-4473-893c-e4a936038c49 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.240141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 399/776] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:34.244849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022d76e-1b15-48e1-b123-a913c0306a49 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.245144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022d76e-1b15-48e1-b123-a913c0306a49 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.245408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022d76e-1b15-48e1-b123-a913c0306a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-a800ba28-f527-4642-88e5-d9296e23b555 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:34.251296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:34.252426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:13:34.253294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:34.256246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 378/777] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:34 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:34.263886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800ba28-f527-4642-88e5-d9296e23b555 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:34.273900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d022d76e-1b15-48e1-b123-a913c0306a49 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:34.275142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d022d76e-1b15-48e1-b123-a913c0306a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:34.343332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:34.363185 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 400/778] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:34.420315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800ba28-f527-4642-88e5-d9296e23b555 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:34.463033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:34.474688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc33ac5-d5a7-4bcc-9862-e085ec134b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:34.478949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc33ac5-d5a7-4bcc-9862-e085ec134b7a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.478949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc33ac5-d5a7-4bcc-9862-e085ec134b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.478949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc33ac5-d5a7-4bcc-9862-e085ec134b7a None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.478949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc33ac5-d5a7-4bcc-9862-e085ec134b7a None admin] RBAC: Adding query filter params (name=network) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:34.478949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc33ac5-d5a7-4bcc-9862-e085ec134b7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:34.478949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc33ac5-d5a7-4bcc-9862-e085ec134b7a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:34.489988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 379/779] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:13:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:34.495836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022d76e-1b15-48e1-b123-a913c0306a49 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:34.611235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022d76e-1b15-48e1-b123-a913c0306a49 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:34.652820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46faf8-eac4-414d-927c-e003845a51da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:34.653903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46faf8-eac4-414d-927c-e003845a51da None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.654400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46faf8-eac4-414d-927c-e003845a51da None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.654886 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46faf8-eac4-414d-927c-e003845a51da None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.655712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46faf8-eac4-414d-927c-e003845a51da None admin] RBAC: Adding query filter params (type=network) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:34.656809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46faf8-eac4-414d-927c-e003845a51da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:34.659224 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46faf8-eac4-414d-927c-e003845a51da None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:34.667045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 401/780] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:13:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:34.672816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc33ac5-d5a7-4bcc-9862-e085ec134b7a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:34.710221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46faf8-eac4-414d-927c-e003845a51da None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:34.771680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:34.831181 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46faf8-eac4-414d-927c-e003845a51da None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:34.904911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc33ac5-d5a7-4bcc-9862-e085ec134b7a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:34.910017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a67b57-e3c2-405d-8737-e49f3bb1358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:34.910667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a67b57-e3c2-405d-8737-e49f3bb1358f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.911074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a67b57-e3c2-405d-8737-e49f3bb1358f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.911346 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a67b57-e3c2-405d-8737-e49f3bb1358f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.931830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a67b57-e3c2-405d-8737-e49f3bb1358f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c9a67b57-e3c2-405d-8737-e49f3bb1358f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:34.938783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:34.939696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:34.940547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:34.943271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 402/781] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:34.944732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff238-76b2-46a3-8644-cc09db368eba None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.945240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff238-76b2-46a3-8644-cc09db368eba None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.945748 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff238-76b2-46a3-8644-cc09db368eba None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.951413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 403/782] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:34.952925 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b2316-6ff2-4a0a-899a-b57cb1d124bd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.953417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b2316-6ff2-4a0a-899a-b57cb1d124bd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.953962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b2316-6ff2-4a0a-899a-b57cb1d124bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.957619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221320e-5e00-4253-8710-36115d1525a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:34.958264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221320e-5e00-4253-8710-36115d1525a7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:34.958588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221320e-5e00-4253-8710-36115d1525a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:34.958859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221320e-5e00-4253-8710-36115d1525a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:34.959560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221320e-5e00-4253-8710-36115d1525a7 None admin] RBAC: Adding query filter params (interface=public, service_id=ec89ba4416324e43b988ab83073dfa95, region_id=RegionOne) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:34.959851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221320e-5e00-4253-8710-36115d1525a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:34.961214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221320e-5e00-4253-8710-36115d1525a7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:34.967919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 380/783] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 1 05:13:34 2022] GET /identity/v3/endpoints?service_id=ec89ba4416324e43b988ab83073dfa95&interface=public®ion_id=RegionOne => generated 194 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:34.969778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221320e-5e00-4253-8710-36115d1525a7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:34.975410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2f2b2316-6ff2-4a0a-899a-b57cb1d124bd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:34.976900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2f2b2316-6ff2-4a0a-899a-b57cb1d124bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:35.041958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:35.057808 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 404/784] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:35.061658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b2316-6ff2-4a0a-899a-b57cb1d124bd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:35.125117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221320e-5e00-4253-8710-36115d1525a7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:35.139965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:35.213130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42292c78-488f-43d4-9400-605ee9801324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:35.213769 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42292c78-488f-43d4-9400-605ee9801324 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:35.214043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42292c78-488f-43d4-9400-605ee9801324 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:35.214308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42292c78-488f-43d4-9400-605ee9801324 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:35.214965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42292c78-488f-43d4-9400-605ee9801324 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:35.215550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42292c78-488f-43d4-9400-605ee9801324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:35.216710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42292c78-488f-43d4-9400-605ee9801324 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:35.225247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b2316-6ff2-4a0a-899a-b57cb1d124bd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:35.239415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 381/785] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:35.246556 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42292c78-488f-43d4-9400-605ee9801324 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:35.270562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f3433-196a-4210-94c7-caf01d8e645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:35.272505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f3433-196a-4210-94c7-caf01d8e645b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:35.273069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f3433-196a-4210-94c7-caf01d8e645b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:35.274757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f3433-196a-4210-94c7-caf01d8e645b None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:35.275593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f3433-196a-4210-94c7-caf01d8e645b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:35.278517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f3433-196a-4210-94c7-caf01d8e645b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:35.288390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 405/786] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 1 05:13:35 2022] GET /identity/v3/services => generated 772 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:35.388044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42292c78-488f-43d4-9400-605ee9801324 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:35.400763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f3433-196a-4210-94c7-caf01d8e645b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:35.409259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:35.466258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acef5a-a2bf-427d-88af-fe00d77ee019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:35.466876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acef5a-a2bf-427d-88af-fe00d77ee019 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:35.467165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acef5a-a2bf-427d-88af-fe00d77ee019 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:35.467440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acef5a-a2bf-427d-88af-fe00d77ee019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:35.487817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acef5a-a2bf-427d-88af-fe00d77ee019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:35.490334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acef5a-a2bf-427d-88af-fe00d77ee019 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-93acef5a-a2bf-427d-88af-fe00d77ee019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:35.507833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:35.508827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:35.509776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:35.509776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:35.509776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:35.509776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 382/787] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:35.516062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acef5a-a2bf-427d-88af-fe00d77ee019 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:35.524700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f3433-196a-4210-94c7-caf01d8e645b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:35.527415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:35.564567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60fc2d-82b7-444e-977d-5be7637cea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:35.565410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60fc2d-82b7-444e-977d-5be7637cea1a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:35.565819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60fc2d-82b7-444e-977d-5be7637cea1a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:35.566191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60fc2d-82b7-444e-977d-5be7637cea1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:35.577805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60fc2d-82b7-444e-977d-5be7637cea1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ce60fc2d-82b7-444e-977d-5be7637cea1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:35.582885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:35.584008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:35.584914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:35.586297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 406/788] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:35.592832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60fc2d-82b7-444e-977d-5be7637cea1a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:35.651566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acef5a-a2bf-427d-88af-fe00d77ee019 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:35.706698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc1e93-9c5b-4baf-adf5-5b365162351f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:35.707377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc1e93-9c5b-4baf-adf5-5b365162351f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:35.707659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc1e93-9c5b-4baf-adf5-5b365162351f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:35.707929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc1e93-9c5b-4baf-adf5-5b365162351f None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:35.708578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc1e93-9c5b-4baf-adf5-5b365162351f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:35.708881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc1e93-9c5b-4baf-adf5-5b365162351f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:35.710293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc1e93-9c5b-4baf-adf5-5b365162351f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:35.735645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 383/789] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:35.740923 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc1e93-9c5b-4baf-adf5-5b365162351f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:35.752513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60fc2d-82b7-444e-977d-5be7637cea1a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:35.789885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:35.923800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc1e93-9c5b-4baf-adf5-5b365162351f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:35.989912 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec781a85-603e-4add-9a00-15c9e00799ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:35.990552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec781a85-603e-4add-9a00-15c9e00799ca None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:35.990831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec781a85-603e-4add-9a00-15c9e00799ca None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:35.991105 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec781a85-603e-4add-9a00-15c9e00799ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:35.999319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec781a85-603e-4add-9a00-15c9e00799ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:36.000972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec781a85-603e-4add-9a00-15c9e00799ca None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:36.005732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920a5ab-f7ed-4634-9550-461e451bf3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:36.006357 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920a5ab-f7ed-4634-9550-461e451bf3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.006635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920a5ab-f7ed-4634-9550-461e451bf3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.006909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920a5ab-f7ed-4634-9550-461e451bf3b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.007582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920a5ab-f7ed-4634-9550-461e451bf3b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:36.007887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920a5ab-f7ed-4634-9550-461e451bf3b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:36.009250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920a5ab-f7ed-4634-9550-461e451bf3b9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-ec781a85-603e-4add-9a00-15c9e00799ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:36.020405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:36.021350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:36.022342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:36.022342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:36.022342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:36.022342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:36.022342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:36.022342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:36.022342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:36.022342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:36.022342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:36.022342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:36.022878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 384/790] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:36.029346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec781a85-603e-4add-9a00-15c9e00799ca None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:36.037626 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 407/791] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:36.042975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920a5ab-f7ed-4634-9550-461e451bf3b9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:36.101839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:36.192076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920a5ab-f7ed-4634-9550-461e451bf3b9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:36.208495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:36.228903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec781a85-603e-4add-9a00-15c9e00799ca None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:36.234077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa0c24-dc9f-4e48-9b56-f25dfbb84ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:36.234935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa0c24-dc9f-4e48-9b56-f25dfbb84ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.235213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa0c24-dc9f-4e48-9b56-f25dfbb84ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.235484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa0c24-dc9f-4e48-9b56-f25dfbb84ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.250762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa0c24-dc9f-4e48-9b56-f25dfbb84ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:36.252634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa0c24-dc9f-4e48-9b56-f25dfbb84ed3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:36.266175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59ad51-7025-49fe-84db-5bb6305c3fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:36.266917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59ad51-7025-49fe-84db-5bb6305c3fdc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.267317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59ad51-7025-49fe-84db-5bb6305c3fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.267688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59ad51-7025-49fe-84db-5bb6305c3fdc None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.268522 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59ad51-7025-49fe-84db-5bb6305c3fdc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:36.268934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59ad51-7025-49fe-84db-5bb6305c3fdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d2fa0c24-dc9f-4e48-9b56-f25dfbb84ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:36.281480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:36.282669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:36.283782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:36.283782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:36.283782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:36.283993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59ad51-7025-49fe-84db-5bb6305c3fdc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:36.284088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 408/792] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:36.284088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57a51-676b-44c9-8b99-313d813ed9c6 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.284088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57a51-676b-44c9-8b99-313d813ed9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.284088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57a51-676b-44c9-8b99-313d813ed9c6 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.284088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 409/793] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:36.286253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b840ca-e6b0-4f17-b03c-6f809148fcf6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.286969 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b840ca-e6b0-4f17-b03c-6f809148fcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.287424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b840ca-e6b0-4f17-b03c-6f809148fcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.297289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 385/794] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 1 05:13:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:36.298209 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59ad51-7025-49fe-84db-5bb6305c3fdc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:36.316783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-32b840ca-e6b0-4f17-b03c-6f809148fcf6 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:36.318654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-32b840ca-e6b0-4f17-b03c-6f809148fcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:36.409491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 410/795] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:36.409491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b840ca-e6b0-4f17-b03c-6f809148fcf6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:36.423092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:36.446265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59ad51-7025-49fe-84db-5bb6305c3fdc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:36.510381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804e8f2-07f6-4dca-b16e-f6f2ff8f9b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:36.511047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e8f2-07f6-4dca-b16e-f6f2ff8f9b23 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.511324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e8f2-07f6-4dca-b16e-f6f2ff8f9b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.511592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e8f2-07f6-4dca-b16e-f6f2ff8f9b23 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.512448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804e8f2-07f6-4dca-b16e-f6f2ff8f9b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:36.512751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804e8f2-07f6-4dca-b16e-f6f2ff8f9b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:36.514168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804e8f2-07f6-4dca-b16e-f6f2ff8f9b23 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:36.530179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 386/796] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:36.536347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804e8f2-07f6-4dca-b16e-f6f2ff8f9b23 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:36.557118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:36.586462 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b840ca-e6b0-4f17-b03c-6f809148fcf6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:36.648386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0df889-5a3e-4012-9095-433ff5af1f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:36.649018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0df889-5a3e-4012-9095-433ff5af1f75 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.649290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0df889-5a3e-4012-9095-433ff5af1f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.649584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0df889-5a3e-4012-9095-433ff5af1f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.660903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0df889-5a3e-4012-9095-433ff5af1f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:36.662401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0df889-5a3e-4012-9095-433ff5af1f75 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9d0df889-5a3e-4012-9095-433ff5af1f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:36.679845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:36.680855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:36.681803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:36.681803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:36.681803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:36.681803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 411/797] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:36.693529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0df889-5a3e-4012-9095-433ff5af1f75 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:36.711730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804e8f2-07f6-4dca-b16e-f6f2ff8f9b23 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:36.748735 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa8646-6ea0-446d-a13b-a3d06801896d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:36.749753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa8646-6ea0-446d-a13b-a3d06801896d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.750493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa8646-6ea0-446d-a13b-a3d06801896d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.751046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa8646-6ea0-446d-a13b-a3d06801896d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.759942 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa8646-6ea0-446d-a13b-a3d06801896d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:36.761884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa8646-6ea0-446d-a13b-a3d06801896d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:36.764814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8cfa8646-6ea0-446d-a13b-a3d06801896d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:36.774360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:36.775901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 387/798] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 1 05:13:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:36.777067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f47f0-47a0-487e-acc6-f7f8b92d1801 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.777944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f47f0-47a0-487e-acc6-f7f8b92d1801 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.777944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f47f0-47a0-487e-acc6-f7f8b92d1801 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.778802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 388/799] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:36.779349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f47f0-47a0-487e-acc6-f7f8b92d1801 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:36.820401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0df889-5a3e-4012-9095-433ff5af1f75 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:36.852610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:36.856875 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29099ab1-a479-4294-a7a3-7d2de4d98fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:36.857512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099ab1-a479-4294-a7a3-7d2de4d98fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.857790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099ab1-a479-4294-a7a3-7d2de4d98fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.858067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099ab1-a479-4294-a7a3-7d2de4d98fd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.858708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29099ab1-a479-4294-a7a3-7d2de4d98fd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:36.859723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29099ab1-a479-4294-a7a3-7d2de4d98fd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:36.861232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29099ab1-a479-4294-a7a3-7d2de4d98fd2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:36.874461 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 412/800] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:36.875746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29099ab1-a479-4294-a7a3-7d2de4d98fd2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:36.901794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f47f0-47a0-487e-acc6-f7f8b92d1801 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:36.936522 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97449995-e2ea-4780-9d60-ec885a2221ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:36.937763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97449995-e2ea-4780-9d60-ec885a2221ec None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.938037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97449995-e2ea-4780-9d60-ec885a2221ec None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.938297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97449995-e2ea-4780-9d60-ec885a2221ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.949686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97449995-e2ea-4780-9d60-ec885a2221ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-97449995-e2ea-4780-9d60-ec885a2221ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:36.954500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:36.957564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 389/801] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:36.958469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2cae2-2e71-4e2a-a5f3-46bc21dfc9dd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:36.959239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2cae2-2e71-4e2a-a5f3-46bc21dfc9dd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:36.959239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2cae2-2e71-4e2a-a5f3-46bc21dfc9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:36.975851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-abc2cae2-2e71-4e2a-a5f3-46bc21dfc9dd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:36.976953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-abc2cae2-2e71-4e2a-a5f3-46bc21dfc9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:36.993357 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29099ab1-a479-4294-a7a3-7d2de4d98fd2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.014417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:37.028288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451c995-8e77-4fee-8849-c83df4f8ddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.028913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451c995-8e77-4fee-8849-c83df4f8ddc0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.032253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451c995-8e77-4fee-8849-c83df4f8ddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.032253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451c995-8e77-4fee-8849-c83df4f8ddc0 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.032253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451c995-8e77-4fee-8849-c83df4f8ddc0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:37.032253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451c995-8e77-4fee-8849-c83df4f8ddc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.032523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451c995-8e77-4fee-8849-c83df4f8ddc0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.061482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 390/802] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:37.061482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2cae2-2e71-4e2a-a5f3-46bc21dfc9dd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.061777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 413/803] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 1 05:13:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:37.061777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451c995-8e77-4fee-8849-c83df4f8ddc0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.130558 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:37.178157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2cae2-2e71-4e2a-a5f3-46bc21dfc9dd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.194291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451c995-8e77-4fee-8849-c83df4f8ddc0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.219898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78700a-e435-4615-93e1-db77404f66ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.220678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78700a-e435-4615-93e1-db77404f66ba None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.221064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78700a-e435-4615-93e1-db77404f66ba None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.221442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78700a-e435-4615-93e1-db77404f66ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.232419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78700a-e435-4615-93e1-db77404f66ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.232419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78700a-e435-4615-93e1-db77404f66ba None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.236634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40ad1-380a-4d58-8da4-dc83fac67173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.237319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40ad1-380a-4d58-8da4-dc83fac67173 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.237698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40ad1-380a-4d58-8da4-dc83fac67173 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.238044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40ad1-380a-4d58-8da4-dc83fac67173 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.238903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea40ad1-380a-4d58-8da4-dc83fac67173 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:37.239292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea40ad1-380a-4d58-8da4-dc83fac67173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.240856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea40ad1-380a-4d58-8da4-dc83fac67173 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cd78700a-e435-4615-93e1-db77404f66ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:37.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:37.246921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:37.247869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:37.247869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:37.247869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:37.247869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:37.247869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:37.247869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:37.247869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:37.247869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:37.254622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 391/804] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:37.255036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 414/805] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:37.255819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40ad1-380a-4d58-8da4-dc83fac67173 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.256276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78700a-e435-4615-93e1-db77404f66ba None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.336357 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:37.368977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78700a-e435-4615-93e1-db77404f66ba None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.382310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40ad1-380a-4d58-8da4-dc83fac67173 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.385569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:37.403074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb610b-b549-4b9c-808b-db2244f2d77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.403696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb610b-b549-4b9c-808b-db2244f2d77f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.403969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb610b-b549-4b9c-808b-db2244f2d77f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.404235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb610b-b549-4b9c-808b-db2244f2d77f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.416175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb610b-b549-4b9c-808b-db2244f2d77f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.417653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb610b-b549-4b9c-808b-db2244f2d77f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.419792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f5b77-4730-4813-8b09-4b7ff0db50ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.420429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f5b77-4730-4813-8b09-4b7ff0db50ce None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.420709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f5b77-4730-4813-8b09-4b7ff0db50ce None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.420981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f5b77-4730-4813-8b09-4b7ff0db50ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.432629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f5b77-4730-4813-8b09-4b7ff0db50ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-adeb610b-b549-4b9c-808b-db2244f2d77f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:37.432765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:37.433943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:37.434858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:37.434858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:37.434858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:37.435039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f5b77-4730-4813-8b09-4b7ff0db50ce None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.439058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 392/806] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:37.439655 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb610b-b549-4b9c-808b-db2244f2d77f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-063f5b77-4730-4813-8b09-4b7ff0db50ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:37.449371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:37.450337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:37.451227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:37.451227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:37.451227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:37.455692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 415/807] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:37.456451 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f5b77-4730-4813-8b09-4b7ff0db50ce None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.549190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb610b-b549-4b9c-808b-db2244f2d77f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.558366 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:37.569890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f5b77-4730-4813-8b09-4b7ff0db50ce None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.584263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebfdf7-ad1b-4aa4-9971-0957ae83dc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.584863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebfdf7-ad1b-4aa4-9971-0957ae83dc9d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.585138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebfdf7-ad1b-4aa4-9971-0957ae83dc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.585404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebfdf7-ad1b-4aa4-9971-0957ae83dc9d None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.586118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebfdf7-ad1b-4aa4-9971-0957ae83dc9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:37.586417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebfdf7-ad1b-4aa4-9971-0957ae83dc9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.587829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebfdf7-ad1b-4aa4-9971-0957ae83dc9d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.605222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 393/808] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:37.605222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebfdf7-ad1b-4aa4-9971-0957ae83dc9d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.613183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2bcf7-8d04-4250-bd19-d9d88539f3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.613929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2bcf7-8d04-4250-bd19-d9d88539f3ad None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.614276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2bcf7-8d04-4250-bd19-d9d88539f3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.614615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2bcf7-8d04-4250-bd19-d9d88539f3ad None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.615282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2bcf7-8d04-4250-bd19-d9d88539f3ad None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.616616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2bcf7-8d04-4250-bd19-d9d88539f3ad None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-02e2bcf7-8d04-4250-bd19-d9d88539f3ad None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:37.626989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:37.628190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:37.629229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 416/809] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:37.630252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2bcf7-8d04-4250-bd19-d9d88539f3ad None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.642257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:37.719974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebfdf7-ad1b-4aa4-9971-0957ae83dc9d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.736421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2bcf7-8d04-4250-bd19-d9d88539f3ad None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.755276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b55683-dfab-4990-85bc-81ebdcbd1e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.755935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b55683-dfab-4990-85bc-81ebdcbd1e8e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.756212 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b55683-dfab-4990-85bc-81ebdcbd1e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.756505 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b55683-dfab-4990-85bc-81ebdcbd1e8e None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.757138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b55683-dfab-4990-85bc-81ebdcbd1e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:37.757442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b55683-dfab-4990-85bc-81ebdcbd1e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.758841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b55683-dfab-4990-85bc-81ebdcbd1e8e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.770292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 394/810] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:37.770727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc6825-7d67-4618-802a-af847b68e5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.771050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b55683-dfab-4990-85bc-81ebdcbd1e8e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.771378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc6825-7d67-4618-802a-af847b68e5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.771662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc6825-7d67-4618-802a-af847b68e5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.771935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc6825-7d67-4618-802a-af847b68e5a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.772573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc6825-7d67-4618-802a-af847b68e5a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:37.772877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc6825-7d67-4618-802a-af847b68e5a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.774265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc6825-7d67-4618-802a-af847b68e5a6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.786578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 417/811] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:37.787455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc6825-7d67-4618-802a-af847b68e5a6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.806689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:37.862156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:37.878031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b55683-dfab-4990-85bc-81ebdcbd1e8e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.897521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc6825-7d67-4618-802a-af847b68e5a6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:37.917289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8f1b6-f070-4ddc-8f65-6cabb08ef28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.918005 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8f1b6-f070-4ddc-8f65-6cabb08ef28e None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.918279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8f1b6-f070-4ddc-8f65-6cabb08ef28e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.918645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8f1b6-f070-4ddc-8f65-6cabb08ef28e None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/b723681faf15434e8529e8fc56c6e9b8/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.920617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8f1b6-f070-4ddc-8f65-6cabb08ef28e None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=b723681faf15434e8529e8fc56c6e9b8, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.922358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8f1b6-f070-4ddc-8f65-6cabb08ef28e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.938882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb9e09-cbca-4d9c-b8f2-6d156d832b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:37.939494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb9e09-cbca-4d9c-b8f2-6d156d832b29 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.939826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb9e09-cbca-4d9c-b8f2-6d156d832b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.940118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb9e09-cbca-4d9c-b8f2-6d156d832b29 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.941074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb9e09-cbca-4d9c-b8f2-6d156d832b29 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:37.941573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb9e09-cbca-4d9c-b8f2-6d156d832b29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:37.943707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb9e09-cbca-4d9c-b8f2-6d156d832b29 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:37.949625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 418/812] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:37.950766 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd390702-2fd1-42a5-8f94-8d02859ad930 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:37.951050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd390702-2fd1-42a5-8f94-8d02859ad930 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:37.951327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd390702-2fd1-42a5-8f94-8d02859ad930 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:37.953777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 419/813] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:37.954822 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd390702-2fd1-42a5-8f94-8d02859ad930 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:37.967025 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 395/814] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:13:37 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/b723681faf15434e8529e8fc56c6e9b8/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:13:37.967855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8f1b6-f070-4ddc-8f65-6cabb08ef28e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:38.066622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:38.085582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8f1b6-f070-4ddc-8f65-6cabb08ef28e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:38.088661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd390702-2fd1-42a5-8f94-8d02859ad930 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:38.130027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8850ce-7bc5-4265-993f-96d763d373ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:38.130275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c7173-cc9b-4c58-9200-a533403e0db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:38.130820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8850ce-7bc5-4265-993f-96d763d373ee None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:38.130918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c7173-cc9b-4c58-9200-a533403e0db9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:38.131168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c7173-cc9b-4c58-9200-a533403e0db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:38.131263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8850ce-7bc5-4265-993f-96d763d373ee None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:38.131442 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c7173-cc9b-4c58-9200-a533403e0db9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:38.131672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8850ce-7bc5-4265-993f-96d763d373ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:38.138806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c7173-cc9b-4c58-9200-a533403e0db9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:38.139043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8850ce-7bc5-4265-993f-96d763d373ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:38.140850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c7173-cc9b-4c58-9200-a533403e0db9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:38.141382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8850ce-7bc5-4265-993f-96d763d373ee None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-700c7173-cc9b-4c58-9200-a533403e0db9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:38.152606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:38.153801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:38.155084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:38.155084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:38.155084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:38.155084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:38.155084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:38.155084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:38.155084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:38.155084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:38.155084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:13:38.155084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-7a8850ce-7bc5-4265-993f-96d763d373ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:38.156006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:38.157181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:38.158380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:38.158380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:38.158380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:38.158380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:38.158380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:38.158380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:38.158380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:38.158380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:38.159030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 420/815] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:38.159030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c7173-cc9b-4c58-9200-a533403e0db9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:38.159203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 396/816] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:38.161036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adc784-bc93-4d74-ba50-e13ea8046635 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:38.162518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adc784-bc93-4d74-ba50-e13ea8046635 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:38.162980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adc784-bc93-4d74-ba50-e13ea8046635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:38.182839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c0adc784-bc93-4d74-ba50-e13ea8046635 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:38.184034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c0adc784-bc93-4d74-ba50-e13ea8046635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:38.186311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:38.268598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 397/817] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:38.268598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adc784-bc93-4d74-ba50-e13ea8046635 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:38.273540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c7173-cc9b-4c58-9200-a533403e0db9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:38.318352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ed93e-0d17-436e-8bc8-abea764dc27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:38.319046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ed93e-0d17-436e-8bc8-abea764dc27a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:38.319333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ed93e-0d17-436e-8bc8-abea764dc27a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:38.319610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ed93e-0d17-436e-8bc8-abea764dc27a None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:38.320638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ed93e-0d17-436e-8bc8-abea764dc27a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:38.321014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ed93e-0d17-436e-8bc8-abea764dc27a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:38.323195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ed93e-0d17-436e-8bc8-abea764dc27a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:38.329060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 421/818] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 1 05:13:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:38.330295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ed93e-0d17-436e-8bc8-abea764dc27a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:38.341447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:38.391512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adc784-bc93-4d74-ba50-e13ea8046635 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:38.438982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685b77e-eea1-4193-8ad2-2faaa68b8620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:38.440180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685b77e-eea1-4193-8ad2-2faaa68b8620 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:38.440869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685b77e-eea1-4193-8ad2-2faaa68b8620 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:38.441246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685b77e-eea1-4193-8ad2-2faaa68b8620 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:38.442388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685b77e-eea1-4193-8ad2-2faaa68b8620 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:38.443148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685b77e-eea1-4193-8ad2-2faaa68b8620 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:38.444980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685b77e-eea1-4193-8ad2-2faaa68b8620 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:38.445527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ed93e-0d17-436e-8bc8-abea764dc27a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:38.469669 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 398/819] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 1 05:13:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:38.470132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:38.473371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685b77e-eea1-4193-8ad2-2faaa68b8620 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:38.488496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45330ec8-dafc-4a65-b941-0fb458ce40fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:38.489173 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330ec8-dafc-4a65-b941-0fb458ce40fa None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:38.489530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330ec8-dafc-4a65-b941-0fb458ce40fa None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:38.489808 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330ec8-dafc-4a65-b941-0fb458ce40fa None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:38.490505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45330ec8-dafc-4a65-b941-0fb458ce40fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:38.490811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45330ec8-dafc-4a65-b941-0fb458ce40fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:38.492358 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45330ec8-dafc-4a65-b941-0fb458ce40fa None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:38.508987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 422/820] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:38.509851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45330ec8-dafc-4a65-b941-0fb458ce40fa None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:38.583785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:38.609641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685b77e-eea1-4193-8ad2-2faaa68b8620 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:38.626715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45330ec8-dafc-4a65-b941-0fb458ce40fa None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:38.650727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d13e6-0939-44b9-ba6b-0fdfa96188d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:38.651626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d13e6-0939-44b9-ba6b-0fdfa96188d2 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:38.651910 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d13e6-0939-44b9-ba6b-0fdfa96188d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:38.652181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d13e6-0939-44b9-ba6b-0fdfa96188d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:38.653010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d13e6-0939-44b9-ba6b-0fdfa96188d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:38.654992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d13e6-0939-44b9-ba6b-0fdfa96188d2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:38.663974 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d646d5-7ccd-48b8-9705-568ea75bb210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:38.664579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d646d5-7ccd-48b8-9705-568ea75bb210 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:38.664846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d646d5-7ccd-48b8-9705-568ea75bb210 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:38.665119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d646d5-7ccd-48b8-9705-568ea75bb210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:38.735891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d646d5-7ccd-48b8-9705-568ea75bb210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:38.737365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d646d5-7ccd-48b8-9705-568ea75bb210 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:38.757544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 399/821] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:13:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:38.757544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d13e6-0939-44b9-ba6b-0fdfa96188d2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:38.826670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:38.832308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:38.885742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 423/822] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:13:38.886792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d646d5-7ccd-48b8-9705-568ea75bb210 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:38.901010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d13e6-0939-44b9-ba6b-0fdfa96188d2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:38.943961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30637ddd-cd2a-4e87-a904-e8bb89aa385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:38.944623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30637ddd-cd2a-4e87-a904-e8bb89aa385c None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:38.944897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30637ddd-cd2a-4e87-a904-e8bb89aa385c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:38.945165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30637ddd-cd2a-4e87-a904-e8bb89aa385c None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/f58aa170e3004197a15dc2dc868d8681/roles/c6689cf082df4be9b69f2208c097f2c3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:38.964010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30637ddd-cd2a-4e87-a904-e8bb89aa385c None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=f58aa170e3004197a15dc2dc868d8681, role_id=c6689cf082df4be9b69f2208c097f2c3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:38.965479 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30637ddd-cd2a-4e87-a904-e8bb89aa385c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:39.009372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d646d5-7ccd-48b8-9705-568ea75bb210 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:39.015631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 400/823] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:13:38 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/f58aa170e3004197a15dc2dc868d8681/roles/c6689cf082df4be9b69f2208c097f2c3 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:13:39.021549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30637ddd-cd2a-4e87-a904-e8bb89aa385c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:39.088286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ededef-fd67-490b-98fa-19572279fe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:39.089206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ededef-fd67-490b-98fa-19572279fe36 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:39.090629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ededef-fd67-490b-98fa-19572279fe36 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:39.091172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ededef-fd67-490b-98fa-19572279fe36 None admin] PATH_INFO: `/v3/services/network` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:39.093240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ededef-fd67-490b-98fa-19572279fe36 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:39.097137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ededef-fd67-490b-98fa-19572279fe36 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-87ededef-fd67-490b-98fa-19572279fe36 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:39.116374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:39.117499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:39.118369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 424/824] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:38 2022] GET /identity/v3/services/network => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:39.125802 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ededef-fd67-490b-98fa-19572279fe36 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:39.182337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:39.186746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30637ddd-cd2a-4e87-a904-e8bb89aa385c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:39.194504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:39.224942 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde132da-c558-4fcd-b1ad-9f67dfe3ffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:39.225743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde132da-c558-4fcd-b1ad-9f67dfe3ffbb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:39.226065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde132da-c558-4fcd-b1ad-9f67dfe3ffbb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:39.226396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde132da-c558-4fcd-b1ad-9f67dfe3ffbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:39.236889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde132da-c558-4fcd-b1ad-9f67dfe3ffbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:39.238506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde132da-c558-4fcd-b1ad-9f67dfe3ffbb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cde132da-c558-4fcd-b1ad-9f67dfe3ffbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:39.253967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:39.255161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:39.256136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:39.256136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:39.256136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:39.256755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 401/825] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:39.261971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde132da-c558-4fcd-b1ad-9f67dfe3ffbb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:39.276509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ededef-fd67-490b-98fa-19572279fe36 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:39.312838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cfb01-4e23-4e35-93fc-f28eba8f52d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:39.314018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cfb01-4e23-4e35-93fc-f28eba8f52d1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:39.314503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cfb01-4e23-4e35-93fc-f28eba8f52d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:39.314970 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cfb01-4e23-4e35-93fc-f28eba8f52d1 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:39.315814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cfb01-4e23-4e35-93fc-f28eba8f52d1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:39.316310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cfb01-4e23-4e35-93fc-f28eba8f52d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:39.317910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cfb01-4e23-4e35-93fc-f28eba8f52d1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:39.323545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 425/826] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:13:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:39.330608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cfb01-4e23-4e35-93fc-f28eba8f52d1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:39.448233 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cfb01-4e23-4e35-93fc-f28eba8f52d1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:39.455399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde132da-c558-4fcd-b1ad-9f67dfe3ffbb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:39.477906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:39.488174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be67ff-5329-4690-8171-e376990a0b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:39.489103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be67ff-5329-4690-8171-e376990a0b39 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:39.489620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be67ff-5329-4690-8171-e376990a0b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:39.490091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be67ff-5329-4690-8171-e376990a0b39 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:39.491366 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be67ff-5329-4690-8171-e376990a0b39 None admin] RBAC: Adding query filter params (type=network) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:39.504236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be67ff-5329-4690-8171-e376990a0b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:39.504236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be67ff-5329-4690-8171-e376990a0b39 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:39.504236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 426/827] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:13:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:39.505515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be67ff-5329-4690-8171-e376990a0b39 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:39.582192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a25ab-e116-4156-89b9-984197db18a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:39.582821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a25ab-e116-4156-89b9-984197db18a7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:39.583102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a25ab-e116-4156-89b9-984197db18a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:39.583375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a25ab-e116-4156-89b9-984197db18a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:39.584037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a25ab-e116-4156-89b9-984197db18a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:39.584340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a25ab-e116-4156-89b9-984197db18a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:39.585758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a25ab-e116-4156-89b9-984197db18a7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:39.623434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 402/828] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:39.626133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be67ff-5329-4690-8171-e376990a0b39 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:39.636841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a25ab-e116-4156-89b9-984197db18a7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:39.653810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:39.694119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23313d-c36d-4204-88ab-d040d08a6e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:39.694740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23313d-c36d-4204-88ab-d040d08a6e9e None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:39.695018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23313d-c36d-4204-88ab-d040d08a6e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:39.695287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23313d-c36d-4204-88ab-d040d08a6e9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:39.695965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23313d-c36d-4204-88ab-d040d08a6e9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:39.697327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23313d-c36d-4204-88ab-d040d08a6e9e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:39.784815 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a25ab-e116-4156-89b9-984197db18a7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:39.801670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 427/829] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:13:39 2022] POST /identity/v3/endpoints => generated 337 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:39.842770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737f07e-94ad-4b6c-9db9-a606f47c1570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:39.844174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737f07e-94ad-4b6c-9db9-a606f47c1570 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:39.844453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737f07e-94ad-4b6c-9db9-a606f47c1570 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:39.844878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737f07e-94ad-4b6c-9db9-a606f47c1570 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:39.857733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737f07e-94ad-4b6c-9db9-a606f47c1570 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:39.859532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737f07e-94ad-4b6c-9db9-a606f47c1570 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e737f07e-94ad-4b6c-9db9-a606f47c1570 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:39.872182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:39.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:39.874186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:39.874186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:39.874186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:39.874186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:39.874186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:39.874186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:39.874186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:39.874186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:39.874186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 403/830] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:39.879712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23313d-c36d-4204-88ab-d040d08a6e9e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:39.913565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:40.044858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23313d-c36d-4204-88ab-d040d08a6e9e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:40.072776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19294b7b-1c48-4668-96fe-d85022643400 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:40.072776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19294b7b-1c48-4668-96fe-d85022643400 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:40.072776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19294b7b-1c48-4668-96fe-d85022643400 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:40.073757 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 404/831] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:40.079869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1438398-f6b8-4341-b083-a76a0e97f465 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:40.080478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1438398-f6b8-4341-b083-a76a0e97f465 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:40.081080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1438398-f6b8-4341-b083-a76a0e97f465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:40.082023 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b304fd-2fd9-45ef-8c13-2f173274d07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:40.082635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b304fd-2fd9-45ef-8c13-2f173274d07b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:40.082911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b304fd-2fd9-45ef-8c13-2f173274d07b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:40.083180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b304fd-2fd9-45ef-8c13-2f173274d07b None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:40.083884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b304fd-2fd9-45ef-8c13-2f173274d07b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:40.084186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b304fd-2fd9-45ef-8c13-2f173274d07b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:40.086120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b304fd-2fd9-45ef-8c13-2f173274d07b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:40.100887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 428/832] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:40.108395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b304fd-2fd9-45ef-8c13-2f173274d07b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:40.110370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f1438398-f6b8-4341-b083-a76a0e97f465 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:40.111581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f1438398-f6b8-4341-b083-a76a0e97f465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:40.217677 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:40.228160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 405/833] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:40.234400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:40.268813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b304fd-2fd9-45ef-8c13-2f173274d07b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:40.305627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4a102-e216-4e0e-83c8-c83c6c588ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:40.306993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4a102-e216-4e0e-83c8-c83c6c588ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:40.307449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4a102-e216-4e0e-83c8-c83c6c588ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:40.307943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4a102-e216-4e0e-83c8-c83c6c588ba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:40.308880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4a102-e216-4e0e-83c8-c83c6c588ba7 None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=7cda3c1f67e54ee08abc014399b07cc5) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:40.309470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4a102-e216-4e0e-83c8-c83c6c588ba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:40.311075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4a102-e216-4e0e-83c8-c83c6c588ba7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:40.318810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 429/834] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:40 2022] GET /identity/v3/role_assignments?user.id=7cda3c1f67e54ee08abc014399b07cc5&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:40.518043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1438398-f6b8-4341-b083-a76a0e97f465 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:40.628677 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1438398-f6b8-4341-b083-a76a0e97f465 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:40.664532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fa9a1-444b-4226-aeab-52d35ec83d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:40.665486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fa9a1-444b-4226-aeab-52d35ec83d1c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:40.665915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fa9a1-444b-4226-aeab-52d35ec83d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:40.666331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fa9a1-444b-4226-aeab-52d35ec83d1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:40.677660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fa9a1-444b-4226-aeab-52d35ec83d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cb4fa9a1-444b-4226-aeab-52d35ec83d1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:40.682699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:40.683843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:13:40.684909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:40.692404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 406/835] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:40.694025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4a102-e216-4e0e-83c8-c83c6c588ba7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:40.802795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4a102-e216-4e0e-83c8-c83c6c588ba7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:40.839098 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed13aa1-7ce5-40b9-8c4a-a755da5784b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:40.839698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed13aa1-7ce5-40b9-8c4a-a755da5784b0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:40.839976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed13aa1-7ce5-40b9-8c4a-a755da5784b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:40.840244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed13aa1-7ce5-40b9-8c4a-a755da5784b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:40.840900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed13aa1-7ce5-40b9-8c4a-a755da5784b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:40.841198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed13aa1-7ce5-40b9-8c4a-a755da5784b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:40.842589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed13aa1-7ce5-40b9-8c4a-a755da5784b0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:40.845802 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:40.852583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 430/836] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:40.858375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fa9a1-444b-4226-aeab-52d35ec83d1c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:40.923597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:40.971190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fa9a1-444b-4226-aeab-52d35ec83d1c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:41.011929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ca037-78f7-4b03-97ff-99450bd0d3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:41.012570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca037-78f7-4b03-97ff-99450bd0d3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.012844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca037-78f7-4b03-97ff-99450bd0d3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.013111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca037-78f7-4b03-97ff-99450bd0d3d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.024318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ca037-78f7-4b03-97ff-99450bd0d3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:41.026201 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ca037-78f7-4b03-97ff-99450bd0d3d0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e86ca037-78f7-4b03-97ff-99450bd0d3d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:41.041969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:41.042980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:41.044319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:41.044319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:41.044319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:41.044319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 407/837] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:41.050890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed13aa1-7ce5-40b9-8c4a-a755da5784b0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:41.053766 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367046c5-5627-40d7-a727-fc83bfe2e1d7 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.054046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367046c5-5627-40d7-a727-fc83bfe2e1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.054317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367046c5-5627-40d7-a727-fc83bfe2e1d7 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.055960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 408/838] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:13:41.060800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8eb35-7523-48ab-945b-5c2621c5512d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.061073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8eb35-7523-48ab-945b-5c2621c5512d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.061346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8eb35-7523-48ab-945b-5c2621c5512d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.080224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e2c8eb35-7523-48ab-945b-5c2621c5512d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:41.081406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e2c8eb35-7523-48ab-945b-5c2621c5512d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:41.170767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed13aa1-7ce5-40b9-8c4a-a755da5784b0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:41.172334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 409/839] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:13:41.209025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db16fac-5cc9-49bf-b4c7-b976e83a61da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:41.210024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db16fac-5cc9-49bf-b4c7-b976e83a61da None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.210533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db16fac-5cc9-49bf-b4c7-b976e83a61da None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.211011 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db16fac-5cc9-49bf-b4c7-b976e83a61da None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.211889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db16fac-5cc9-49bf-b4c7-b976e83a61da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:41.212404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db16fac-5cc9-49bf-b4c7-b976e83a61da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:41.213997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db16fac-5cc9-49bf-b4c7-b976e83a61da None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:41.231566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 431/840] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:41.233188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8eb35-7523-48ab-945b-5c2621c5512d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:41.256811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424d6aa-7516-47c1-bb70-b89c61ec7540 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.257122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424d6aa-7516-47c1-bb70-b89c61ec7540 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.257369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424d6aa-7516-47c1-bb70-b89c61ec7540 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.258979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 432/841] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:41.263906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d21c5-6172-401d-ab65-942af30ed0fd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.264187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d21c5-6172-401d-ab65-942af30ed0fd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.264463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d21c5-6172-401d-ab65-942af30ed0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.282437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:41.284751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b61d21c5-6172-401d-ab65-942af30ed0fd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:41.286143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b61d21c5-6172-401d-ab65-942af30ed0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:41.360279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8eb35-7523-48ab-945b-5c2621c5512d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:41.361569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:41.378378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 433/842] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:13:41.402180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f8f22-3871-43ac-9b3c-1311d399c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:41.404123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f8f22-3871-43ac-9b3c-1311d399c606 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.404607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f8f22-3871-43ac-9b3c-1311d399c606 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.405078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f8f22-3871-43ac-9b3c-1311d399c606 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.413396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f8f22-3871-43ac-9b3c-1311d399c606 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:41.415128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f8f22-3871-43ac-9b3c-1311d399c606 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-a56f8f22-3871-43ac-9b3c-1311d399c606 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:41.427880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:41.428972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:41.429973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:41.429973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:41.429973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:41.429973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:41.429973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:41.429973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:41.429973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:41.429973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:41.429973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:13:41.429973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:41.437535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 410/843] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:13:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:41.439153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d21c5-6172-401d-ab65-942af30ed0fd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:41.450503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f8f22-3871-43ac-9b3c-1311d399c606 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:41.557237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d21c5-6172-401d-ab65-942af30ed0fd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:41.566897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:41.571476 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f8f22-3871-43ac-9b3c-1311d399c606 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:41.593841 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dff05-c192-4e43-9021-791c5e09e25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:41.594664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dff05-c192-4e43-9021-791c5e09e25a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.594949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dff05-c192-4e43-9021-791c5e09e25a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.595225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dff05-c192-4e43-9021-791c5e09e25a None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.595954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777dff05-c192-4e43-9021-791c5e09e25a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:41.596318 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777dff05-c192-4e43-9021-791c5e09e25a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:41.597704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777dff05-c192-4e43-9021-791c5e09e25a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:41.611267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f361b22-4692-4113-b492-1f07a8eb75ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:41.612186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f361b22-4692-4113-b492-1f07a8eb75ad None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.612707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f361b22-4692-4113-b492-1f07a8eb75ad None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.613181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f361b22-4692-4113-b492-1f07a8eb75ad None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.613993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f361b22-4692-4113-b492-1f07a8eb75ad None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:41.615581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f361b22-4692-4113-b492-1f07a8eb75ad None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:41.618433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 434/844] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 1 05:13:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:41.626037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dff05-c192-4e43-9021-791c5e09e25a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-7f361b22-4692-4113-b492-1f07a8eb75ad None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:41.628699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:41.629961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:13:41.631033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:41.633620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 411/845] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:13:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:41.639962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f361b22-4692-4113-b492-1f07a8eb75ad None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:41.679755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:41.744124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dff05-c192-4e43-9021-791c5e09e25a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:41.752969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f361b22-4692-4113-b492-1f07a8eb75ad None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:41.783101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf173398-bce6-4a3e-a815-befd86b61986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:41.783790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf173398-bce6-4a3e-a815-befd86b61986 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.784071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf173398-bce6-4a3e-a815-befd86b61986 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.784553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf173398-bce6-4a3e-a815-befd86b61986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.789338 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fd02f-11ee-47e7-8a5b-6c9c41471e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:41.789975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd02f-11ee-47e7-8a5b-6c9c41471e95 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.790252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd02f-11ee-47e7-8a5b-6c9c41471e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.790520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd02f-11ee-47e7-8a5b-6c9c41471e95 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.791151 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fd02f-11ee-47e7-8a5b-6c9c41471e95 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:41.791455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fd02f-11ee-47e7-8a5b-6c9c41471e95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:41.792826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fd02f-11ee-47e7-8a5b-6c9c41471e95 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:41.797758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 412/846] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:13:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:41.798593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fd02f-11ee-47e7-8a5b-6c9c41471e95 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:41.830797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:41.868953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf173398-bce6-4a3e-a815-befd86b61986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:41.870480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf173398-bce6-4a3e-a815-befd86b61986 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:41.916064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fd02f-11ee-47e7-8a5b-6c9c41471e95 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:41.927162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:41.952849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07d389-3fe1-40a6-9d7c-85f3dc71ad5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:41.953500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07d389-3fe1-40a6-9d7c-85f3dc71ad5a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:41.953780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07d389-3fe1-40a6-9d7c-85f3dc71ad5a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:41.954047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07d389-3fe1-40a6-9d7c-85f3dc71ad5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:41.965625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07d389-3fe1-40a6-9d7c-85f3dc71ad5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-6a07d389-3fe1-40a6-9d7c-85f3dc71ad5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:41.970557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:41.971710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:13:41.972781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:41.976315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 413/847] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:13:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:41.977128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07d389-3fe1-40a6-9d7c-85f3dc71ad5a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:41.996434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 435/848] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 05:13:41.997542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf173398-bce6-4a3e-a815-befd86b61986 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:42.077076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:42.091883 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07d389-3fe1-40a6-9d7c-85f3dc71ad5a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:42.110481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf173398-bce6-4a3e-a815-befd86b61986 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:42.128154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b67ff5-5dcd-4d07-8ea6-f2b070c84345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:42.128998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b67ff5-5dcd-4d07-8ea6-f2b070c84345 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.129440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b67ff5-5dcd-4d07-8ea6-f2b070c84345 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.129742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b67ff5-5dcd-4d07-8ea6-f2b070c84345 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.130374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b67ff5-5dcd-4d07-8ea6-f2b070c84345 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:42.130678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b67ff5-5dcd-4d07-8ea6-f2b070c84345 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:42.132054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b67ff5-5dcd-4d07-8ea6-f2b070c84345 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:42.145490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 414/849] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:13:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:42.145490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b67ff5-5dcd-4d07-8ea6-f2b070c84345 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:42.161622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ccf84-881e-4786-b463-fb75ff9bce7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:42.162362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ccf84-881e-4786-b463-fb75ff9bce7d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.163263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ccf84-881e-4786-b463-fb75ff9bce7d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.163544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ccf84-881e-4786-b463-fb75ff9bce7d None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.164564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ccf84-881e-4786-b463-fb75ff9bce7d None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:42.164878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ccf84-881e-4786-b463-fb75ff9bce7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:42.166522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ccf84-881e-4786-b463-fb75ff9bce7d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:42.180007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 436/850] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:13:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:42.185603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ccf84-881e-4786-b463-fb75ff9bce7d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:42.201362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:42.265819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b67ff5-5dcd-4d07-8ea6-f2b070c84345 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:42.306147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa9e63-bce1-43e2-819f-97b25a48d12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:42.306797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa9e63-bce1-43e2-819f-97b25a48d12f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.307058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa9e63-bce1-43e2-819f-97b25a48d12f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.307378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa9e63-bce1-43e2-819f-97b25a48d12f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.320085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa9e63-bce1-43e2-819f-97b25a48d12f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:42.321584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa9e63-bce1-43e2-819f-97b25a48d12f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:42.328753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ccf84-881e-4786-b463-fb75ff9bce7d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-07fa9e63-bce1-43e2-819f-97b25a48d12f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:42.338348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:42.339339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:42.340366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:42.340366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:42.340366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:42.340366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 415/851] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:42.347194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa9e63-bce1-43e2-819f-97b25a48d12f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:42.350486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:42.371496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6392d87-5cfd-44b6-a4e7-d3e42052abf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:42.373015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6392d87-5cfd-44b6-a4e7-d3e42052abf4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.373449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6392d87-5cfd-44b6-a4e7-d3e42052abf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.373903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6392d87-5cfd-44b6-a4e7-d3e42052abf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.386583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6392d87-5cfd-44b6-a4e7-d3e42052abf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:42.388103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6392d87-5cfd-44b6-a4e7-d3e42052abf4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c6392d87-5cfd-44b6-a4e7-d3e42052abf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:42.403479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:42.404645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:42.405845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:42.405845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:42.405845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:42.411512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 437/852] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:42.411512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ed4b0-ff2e-4c8c-a788-787bf51d1e46 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.411512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ed4b0-ff2e-4c8c-a788-787bf51d1e46 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.411512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ed4b0-ff2e-4c8c-a788-787bf51d1e46 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.413436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 438/853] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:42.414223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ed4b0-ff2e-4c8c-a788-787bf51d1e46 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:42.466401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa9e63-bce1-43e2-819f-97b25a48d12f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:42.479635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:42.504497 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e6563-192c-4ab5-aa09-2f51bb3bea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:42.505340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e6563-192c-4ab5-aa09-2f51bb3bea1a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.505824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e6563-192c-4ab5-aa09-2f51bb3bea1a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.506157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e6563-192c-4ab5-aa09-2f51bb3bea1a None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.507014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e6563-192c-4ab5-aa09-2f51bb3bea1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:42.507379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e6563-192c-4ab5-aa09-2f51bb3bea1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:42.509063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e6563-192c-4ab5-aa09-2f51bb3bea1a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:42.525810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ed4b0-ff2e-4c8c-a788-787bf51d1e46 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:42.529393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 416/854] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:42.531474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea9302-5fcb-45ee-a2ea-0f4443278b3c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.531784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea9302-5fcb-45ee-a2ea-0f4443278b3c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.532052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea9302-5fcb-45ee-a2ea-0f4443278b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.551211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8eea9302-5fcb-45ee-a2ea-0f4443278b3c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:42.552800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8eea9302-5fcb-45ee-a2ea-0f4443278b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:42.567320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54d476-c3dc-43ed-8636-4fb99fa40c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:42.567934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54d476-c3dc-43ed-8636-4fb99fa40c64 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.568214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54d476-c3dc-43ed-8636-4fb99fa40c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.568487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54d476-c3dc-43ed-8636-4fb99fa40c64 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.569131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54d476-c3dc-43ed-8636-4fb99fa40c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:42.569432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54d476-c3dc-43ed-8636-4fb99fa40c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:42.570887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54d476-c3dc-43ed-8636-4fb99fa40c64 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:42.583563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 439/855] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:42.584389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54d476-c3dc-43ed-8636-4fb99fa40c64 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:42.625393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:42.635766 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 417/856] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:42.652537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea9302-5fcb-45ee-a2ea-0f4443278b3c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:42.714102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54d476-c3dc-43ed-8636-4fb99fa40c64 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:42.735009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:42.750421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d006d5-b4b1-43b1-860e-e38b0d739e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:42.751403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d006d5-b4b1-43b1-860e-e38b0d739e5a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.751946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d006d5-b4b1-43b1-860e-e38b0d739e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.752471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d006d5-b4b1-43b1-860e-e38b0d739e5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.753314 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea9302-5fcb-45ee-a2ea-0f4443278b3c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:42.760644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d006d5-b4b1-43b1-860e-e38b0d739e5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:42.762843 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d006d5-b4b1-43b1-860e-e38b0d739e5a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-b9d006d5-b4b1-43b1-860e-e38b0d739e5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:42.775341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:42.776239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:42.777196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:42.777196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:42.777196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:42.777196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:42.777196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:42.777196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:42.777196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:42.777196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:42.779367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 440/857] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:42.786134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d006d5-b4b1-43b1-860e-e38b0d739e5a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:42.790863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2243a1-66df-43d6-8bf0-3a6cfea4d1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:42.791010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2243a1-66df-43d6-8bf0-3a6cfea4d1df None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.791258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2243a1-66df-43d6-8bf0-3a6cfea4d1df None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.791527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2243a1-66df-43d6-8bf0-3a6cfea4d1df None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.799650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2243a1-66df-43d6-8bf0-3a6cfea4d1df None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:42.801154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2243a1-66df-43d6-8bf0-3a6cfea4d1df None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-0d2243a1-66df-43d6-8bf0-3a6cfea4d1df None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:42.813483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:42.814470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:42.815444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:42.815444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:42.815444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:42.815444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:42.815444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:42.815444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:42.815444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:42.815444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:42.815444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:42.815444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:42.819383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 418/858] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 1 05:13:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:42.820261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2243a1-66df-43d6-8bf0-3a6cfea4d1df None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:42.880519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:42.927240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d006d5-b4b1-43b1-860e-e38b0d739e5a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:42.931725 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2243a1-66df-43d6-8bf0-3a6cfea4d1df None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:42.964061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0d5d7-2a72-4ee1-8471-cdd4a333f9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:42.964670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0d5d7-2a72-4ee1-8471-cdd4a333f9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.964951 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0d5d7-2a72-4ee1-8471-cdd4a333f9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.965223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0d5d7-2a72-4ee1-8471-cdd4a333f9a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.965938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0d5d7-2a72-4ee1-8471-cdd4a333f9a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:42.966246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0d5d7-2a72-4ee1-8471-cdd4a333f9a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:42.967296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8a90e-77bc-4be5-84ca-cbf6b42c3881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:42.967665 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0d5d7-2a72-4ee1-8471-cdd4a333f9a7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:42.967933 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8a90e-77bc-4be5-84ca-cbf6b42c3881 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:42.968782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8a90e-77bc-4be5-84ca-cbf6b42c3881 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:42.969015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8a90e-77bc-4be5-84ca-cbf6b42c3881 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:42.969959 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8a90e-77bc-4be5-84ca-cbf6b42c3881 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:42.970263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8a90e-77bc-4be5-84ca-cbf6b42c3881 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:42.971831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8a90e-77bc-4be5-84ca-cbf6b42c3881 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:42.985016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 441/859] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:42.985016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0d5d7-2a72-4ee1-8471-cdd4a333f9a7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:43.000807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 419/860] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 1 05:13:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:43.000807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8a90e-77bc-4be5-84ca-cbf6b42c3881 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:43.032214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:43.095268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0d5d7-2a72-4ee1-8471-cdd4a333f9a7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:43.106769 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8a90e-77bc-4be5-84ca-cbf6b42c3881 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:43.118757 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:43.132167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82dd2a-f9a2-4169-b74c-2d5e975a0c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:43.132732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82dd2a-f9a2-4169-b74c-2d5e975a0c17 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:43.133007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82dd2a-f9a2-4169-b74c-2d5e975a0c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:43.133276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82dd2a-f9a2-4169-b74c-2d5e975a0c17 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:43.133885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82dd2a-f9a2-4169-b74c-2d5e975a0c17 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:43.135247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82dd2a-f9a2-4169-b74c-2d5e975a0c17 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9e82dd2a-f9a2-4169-b74c-2d5e975a0c17 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:43.144973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:43.146127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:43.147638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 442/861] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Sep 1 05:13:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:43.148458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82dd2a-f9a2-4169-b74c-2d5e975a0c17 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:43.154570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e422e0-a2c8-475f-a165-218c412208ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:43.155184 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e422e0-a2c8-475f-a165-218c412208ff None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:43.155505 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e422e0-a2c8-475f-a165-218c412208ff None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:43.155768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e422e0-a2c8-475f-a165-218c412208ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:43.156496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e422e0-a2c8-475f-a165-218c412208ff None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=b723681faf15434e8529e8fc56c6e9b8) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:43.156805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e422e0-a2c8-475f-a165-218c412208ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:43.158736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e422e0-a2c8-475f-a165-218c412208ff None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:43.173573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 420/862] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:42 2022] GET /identity/v3/role_assignments?user.id=b723681faf15434e8529e8fc56c6e9b8&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:43.174580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e422e0-a2c8-475f-a165-218c412208ff None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:43.278009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82dd2a-f9a2-4169-b74c-2d5e975a0c17 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:43.281937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:43.312276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e422e0-a2c8-475f-a165-218c412208ff None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:43.321996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4649b0-7379-4163-bae0-b2ee6b6ddba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:43.323196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4649b0-7379-4163-bae0-b2ee6b6ddba1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:43.323697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4649b0-7379-4163-bae0-b2ee6b6ddba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:43.324164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4649b0-7379-4163-bae0-b2ee6b6ddba1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:43.356839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68697ecb-09e2-4ad0-b493-e7a8009ba2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:43.357447 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68697ecb-09e2-4ad0-b493-e7a8009ba2cb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:43.357773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68697ecb-09e2-4ad0-b493-e7a8009ba2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:43.358042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68697ecb-09e2-4ad0-b493-e7a8009ba2cb None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:43.358795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68697ecb-09e2-4ad0-b493-e7a8009ba2cb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:43.359101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68697ecb-09e2-4ad0-b493-e7a8009ba2cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:43.360467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68697ecb-09e2-4ad0-b493-e7a8009ba2cb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:43.367377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 421/863] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:13:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:43.372192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68697ecb-09e2-4ad0-b493-e7a8009ba2cb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:43.406254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4649b0-7379-4163-bae0-b2ee6b6ddba1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:43.408454 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4649b0-7379-4163-bae0-b2ee6b6ddba1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:43.425578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:43.504510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68697ecb-09e2-4ad0-b493-e7a8009ba2cb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:43.531821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 443/864] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:13:43.536921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4649b0-7379-4163-bae0-b2ee6b6ddba1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:43.540694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:43.545261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85169edc-9a11-4045-a8a9-8688540a8984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:43.546802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85169edc-9a11-4045-a8a9-8688540a8984 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:43.547446 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85169edc-9a11-4045-a8a9-8688540a8984 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:43.547948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85169edc-9a11-4045-a8a9-8688540a8984 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:43.548741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85169edc-9a11-4045-a8a9-8688540a8984 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:43.549207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85169edc-9a11-4045-a8a9-8688540a8984 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:43.556374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85169edc-9a11-4045-a8a9-8688540a8984 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:43.556374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 422/865] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:13:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:43.657139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4649b0-7379-4163-bae0-b2ee6b6ddba1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:43.695590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed32247-b6b5-43f9-acf9-6b23f9435907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:43.696250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed32247-b6b5-43f9-acf9-6b23f9435907 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:43.696530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed32247-b6b5-43f9-acf9-6b23f9435907 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:43.696802 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed32247-b6b5-43f9-acf9-6b23f9435907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:43.709125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed32247-b6b5-43f9-acf9-6b23f9435907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:43.719051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed32247-b6b5-43f9-acf9-6b23f9435907 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9ed32247-b6b5-43f9-acf9-6b23f9435907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:43.739696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:43.740703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:43.741677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:43.741677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:43.741677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:43.741677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 444/866] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:43.747060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85169edc-9a11-4045-a8a9-8688540a8984 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:43.856055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85169edc-9a11-4045-a8a9-8688540a8984 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:43.857620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:43.896865 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777b4e3-eabf-4ba8-9472-3b6eb29d646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:43.897812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777b4e3-eabf-4ba8-9472-3b6eb29d646b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:43.898272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777b4e3-eabf-4ba8-9472-3b6eb29d646b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:43.898802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777b4e3-eabf-4ba8-9472-3b6eb29d646b None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:43.899626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777b4e3-eabf-4ba8-9472-3b6eb29d646b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:43.900106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777b4e3-eabf-4ba8-9472-3b6eb29d646b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:43.901677 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777b4e3-eabf-4ba8-9472-3b6eb29d646b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:43.914480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 423/867] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:43.925585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed32247-b6b5-43f9-acf9-6b23f9435907 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:43.986599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:44.034201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed32247-b6b5-43f9-acf9-6b23f9435907 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:44.070099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b338b2-3d73-4dab-bb4b-6b78366fb6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:44.070973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b338b2-3d73-4dab-bb4b-6b78366fb6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:44.071395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b338b2-3d73-4dab-bb4b-6b78366fb6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:44.071835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b338b2-3d73-4dab-bb4b-6b78366fb6e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:44.079786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b338b2-3d73-4dab-bb4b-6b78366fb6e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:44.081403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b338b2-3d73-4dab-bb4b-6b78366fb6e6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-41b338b2-3d73-4dab-bb4b-6b78366fb6e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:44.094096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:44.095212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:44.096324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:44.096324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:44.096324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:44.096324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:44.096324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:44.096324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:44.096324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:44.096324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:44.098333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 445/868] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:44.104215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777b4e3-eabf-4ba8-9472-3b6eb29d646b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:44.152286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea8c95-b20c-4ee4-abae-f3c118ba0f65 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:44.152616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea8c95-b20c-4ee4-abae-f3c118ba0f65 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:44.152825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea8c95-b20c-4ee4-abae-f3c118ba0f65 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:44.154564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 446/869] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:13:44.162296 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b93221-5e38-498c-92b3-d16c2a09ef4c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:44.162563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b93221-5e38-498c-92b3-d16c2a09ef4c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:44.162838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b93221-5e38-498c-92b3-d16c2a09ef4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:44.188031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-09b93221-5e38-498c-92b3-d16c2a09ef4c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:44.189779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-09b93221-5e38-498c-92b3-d16c2a09ef4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:44.225663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777b4e3-eabf-4ba8-9472-3b6eb29d646b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:44.263999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ec33d-38dd-4ebe-998b-d19102fc7f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:44.265610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ec33d-38dd-4ebe-998b-d19102fc7f1f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:44.266274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ec33d-38dd-4ebe-998b-d19102fc7f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:44.266802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ec33d-38dd-4ebe-998b-d19102fc7f1f None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:44.267719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ec33d-38dd-4ebe-998b-d19102fc7f1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:44.268280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ec33d-38dd-4ebe-998b-d19102fc7f1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:44.269962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ec33d-38dd-4ebe-998b-d19102fc7f1f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:44.283749 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 424/870] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:44.284648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 447/871] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:44.293478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ec33d-38dd-4ebe-998b-d19102fc7f1f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:44.316877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:44.397201 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ec33d-38dd-4ebe-998b-d19102fc7f1f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:44.434384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d9771-dbec-465f-868d-48307dd490c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:44.434982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d9771-dbec-465f-868d-48307dd490c1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:44.435254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d9771-dbec-465f-868d-48307dd490c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:44.435518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d9771-dbec-465f-868d-48307dd490c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:44.436227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d9771-dbec-465f-868d-48307dd490c1 None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=f58aa170e3004197a15dc2dc868d8681) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:44.437126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d9771-dbec-465f-868d-48307dd490c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:44.438632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d9771-dbec-465f-868d-48307dd490c1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:44.444805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 425/872] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:44 2022] GET /identity/v3/role_assignments?user.id=f58aa170e3004197a15dc2dc868d8681&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:44.570833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b93221-5e38-498c-92b3-d16c2a09ef4c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:44.620946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:44.681326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b93221-5e38-498c-92b3-d16c2a09ef4c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:44.735803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31e61c-f7eb-438d-9ece-e24ad76b3273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:44.736421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31e61c-f7eb-438d-9ece-e24ad76b3273 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:44.736703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31e61c-f7eb-438d-9ece-e24ad76b3273 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:44.737015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31e61c-f7eb-438d-9ece-e24ad76b3273 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:44.737733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31e61c-f7eb-438d-9ece-e24ad76b3273 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:44.739257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31e61c-f7eb-438d-9ece-e24ad76b3273 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:44.819813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 448/873] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:13:44 2022] POST /identity/v3/services => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:45.275310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17039012-537e-4e91-8bd1-07cc8c371ea3 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:45.275835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17039012-537e-4e91-8bd1-07cc8c371ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:45.275835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17039012-537e-4e91-8bd1-07cc8c371ea3 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:45.277408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 426/874] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:45.282838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823913f9-92b3-44f5-86ba-b4cf38a132e9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:45.283122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823913f9-92b3-44f5-86ba-b4cf38a132e9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:45.283396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823913f9-92b3-44f5-86ba-b4cf38a132e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:45.301472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-823913f9-92b3-44f5-86ba-b4cf38a132e9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:45.302688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-823913f9-92b3-44f5-86ba-b4cf38a132e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:45.382507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 449/875] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:45.587794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d14c9-c3e6-4438-a569-c3e7db3f9c2d None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:45.588101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d14c9-c3e6-4438-a569-c3e7db3f9c2d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:45.588352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d14c9-c3e6-4438-a569-c3e7db3f9c2d None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:45.589981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 427/876] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:45.596584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6c2f8-e8ae-47e5-badb-8edd876300b1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:45.597435 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6c2f8-e8ae-47e5-badb-8edd876300b1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:45.597944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6c2f8-e8ae-47e5-badb-8edd876300b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:45.617500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a5c6c2f8-e8ae-47e5-badb-8edd876300b1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:45.618707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a5c6c2f8-e8ae-47e5-badb-8edd876300b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:45.651139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:45.693785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 450/877] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:45.694006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d14c9-c3e6-4438-a569-c3e7db3f9c2d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:45.785033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:45.801261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d14c9-c3e6-4438-a569-c3e7db3f9c2d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:45.838449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed7fcb-5a30-4197-9ce9-5ab7054d4356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:45.839000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed7fcb-5a30-4197-9ce9-5ab7054d4356 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:45.839281 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed7fcb-5a30-4197-9ce9-5ab7054d4356 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:45.839553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed7fcb-5a30-4197-9ce9-5ab7054d4356 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:45.840138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed7fcb-5a30-4197-9ce9-5ab7054d4356 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:45.841544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed7fcb-5a30-4197-9ce9-5ab7054d4356 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8eed7fcb-5a30-4197-9ce9-5ab7054d4356 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:45.850682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:45.851689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:45.852572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 428/878] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:13:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:45.857908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6c2f8-e8ae-47e5-badb-8edd876300b1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:45.962467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6c2f8-e8ae-47e5-badb-8edd876300b1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:45.998339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed7fcb-5a30-4197-9ce9-5ab7054d4356 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:46.002911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0871a-8c95-4f11-a17a-73170a87819f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:46.003606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0871a-8c95-4f11-a17a-73170a87819f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:46.003885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0871a-8c95-4f11-a17a-73170a87819f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:46.004158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0871a-8c95-4f11-a17a-73170a87819f None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:46.004801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0871a-8c95-4f11-a17a-73170a87819f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:46.005105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0871a-8c95-4f11-a17a-73170a87819f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:46.006512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0871a-8c95-4f11-a17a-73170a87819f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:46.012125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 451/879] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 1 05:13:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:46.018196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0871a-8c95-4f11-a17a-73170a87819f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:46.065878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:46.129328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0871a-8c95-4f11-a17a-73170a87819f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:46.133693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed7fcb-5a30-4197-9ce9-5ab7054d4356 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:46.169143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989ac12-8a90-4bdf-b225-269115336f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:46.169794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989ac12-8a90-4bdf-b225-269115336f78 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:46.170164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989ac12-8a90-4bdf-b225-269115336f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:46.170439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989ac12-8a90-4bdf-b225-269115336f78 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:46.171135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989ac12-8a90-4bdf-b225-269115336f78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:46.172862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989ac12-8a90-4bdf-b225-269115336f78 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:46.174176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e81ff-3546-4a01-9a14-74249a7cd01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:46.174385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e81ff-3546-4a01-9a14-74249a7cd01e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:46.174911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e81ff-3546-4a01-9a14-74249a7cd01e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:46.174911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e81ff-3546-4a01-9a14-74249a7cd01e None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:46.175866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e81ff-3546-4a01-9a14-74249a7cd01e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:46.175866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e81ff-3546-4a01-9a14-74249a7cd01e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:46.177153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e81ff-3546-4a01-9a14-74249a7cd01e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:46.182960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 452/880] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 1 05:13:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:46.284850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 429/881] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:13:45 2022] POST /identity/v3/services => generated 269 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:13:46.532349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d440a-fb2e-4dd8-a745-d0b7abc508ca None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:46.532543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d440a-fb2e-4dd8-a745-d0b7abc508ca None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:46.532816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d440a-fb2e-4dd8-a745-d0b7abc508ca None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:46.534450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 453/882] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:46.539964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c4530-bb89-4939-8e08-9a68e74285a6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:46.540248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c4530-bb89-4939-8e08-9a68e74285a6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:46.540520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c4530-bb89-4939-8e08-9a68e74285a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:46.559788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-666c4530-bb89-4939-8e08-9a68e74285a6 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:46.560977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-666c4530-bb89-4939-8e08-9a68e74285a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:46.564233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:46.639342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 430/883] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:46.842138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a5b90-0fdd-4e27-aab6-9690c80ee5bb None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:46.843428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a5b90-0fdd-4e27-aab6-9690c80ee5bb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:46.843951 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a5b90-0fdd-4e27-aab6-9690c80ee5bb None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:46.845817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 454/884] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:46.850782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4195405-7028-4cb8-8c18-f8cb10f2fd39 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:46.851307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4195405-7028-4cb8-8c18-f8cb10f2fd39 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:46.851771 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4195405-7028-4cb8-8c18-f8cb10f2fd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:46.870654 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f4195405-7028-4cb8-8c18-f8cb10f2fd39 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:46.872342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f4195405-7028-4cb8-8c18-f8cb10f2fd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:46.929048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a5b90-0fdd-4e27-aab6-9690c80ee5bb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:46.946360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 431/885] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:47.033611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:47.043808 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a5b90-0fdd-4e27-aab6-9690c80ee5bb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:47.083489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecab12-87d1-4635-8df5-087cd4deec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:47.084469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecab12-87d1-4635-8df5-087cd4deec6e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:47.084945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecab12-87d1-4635-8df5-087cd4deec6e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:47.085432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecab12-87d1-4635-8df5-087cd4deec6e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:47.096586 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecab12-87d1-4635-8df5-087cd4deec6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-40ecab12-87d1-4635-8df5-087cd4deec6e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:47.101708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:47.102629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:13:47.103497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:47.106121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 455/886] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 1 05:13:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:47.114541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4195405-7028-4cb8-8c18-f8cb10f2fd39 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:47.182126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:47.218394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4195405-7028-4cb8-8c18-f8cb10f2fd39 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:47.251460 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecab12-87d1-4635-8df5-087cd4deec6e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:47.254965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df011b61-a53c-4910-a744-257e25208938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:47.255624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df011b61-a53c-4910-a744-257e25208938 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:47.256840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df011b61-a53c-4910-a744-257e25208938 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:47.257117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df011b61-a53c-4910-a744-257e25208938 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:47.257817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df011b61-a53c-4910-a744-257e25208938 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:47.258232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df011b61-a53c-4910-a744-257e25208938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:47.259640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df011b61-a53c-4910-a744-257e25208938 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:47.271073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 432/887] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 1 05:13:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:47.276526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df011b61-a53c-4910-a744-257e25208938 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:47.379348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecab12-87d1-4635-8df5-087cd4deec6e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:47.398131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df011b61-a53c-4910-a744-257e25208938 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:47.420139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca7c63-86c4-47ed-849d-5d18c5c170eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:47.420872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca7c63-86c4-47ed-849d-5d18c5c170eb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:47.421257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca7c63-86c4-47ed-849d-5d18c5c170eb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:47.421607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca7c63-86c4-47ed-849d-5d18c5c170eb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:47.422205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca7c63-86c4-47ed-849d-5d18c5c170eb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:47.423664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca7c63-86c4-47ed-849d-5d18c5c170eb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:47.432692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-39ca7c63-86c4-47ed-849d-5d18c5c170eb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:47.434446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:47.435587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:47.436676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 456/888] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:13:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:47.442119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca7c63-86c4-47ed-849d-5d18c5c170eb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:47.446117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06a66d-3f2f-4807-951a-b5000593222a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:47.447913 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06a66d-3f2f-4807-951a-b5000593222a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:47.448194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06a66d-3f2f-4807-951a-b5000593222a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:47.448460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06a66d-3f2f-4807-951a-b5000593222a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:47.460934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:47.463347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06a66d-3f2f-4807-951a-b5000593222a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:47.465295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06a66d-3f2f-4807-951a-b5000593222a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-7f06a66d-3f2f-4807-951a-b5000593222a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:47.482729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:47.483988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:47.485864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:47.485864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:47.485864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:47.485864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 433/889] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:47.496763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06a66d-3f2f-4807-951a-b5000593222a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:47.570359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca7c63-86c4-47ed-849d-5d18c5c170eb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:47.609411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dcf7d-d4c2-45c7-9537-567569cc9fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:47.610238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dcf7d-d4c2-45c7-9537-567569cc9fcc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:47.610521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dcf7d-d4c2-45c7-9537-567569cc9fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:47.610795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dcf7d-d4c2-45c7-9537-567569cc9fcc None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:47.611434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630dcf7d-d4c2-45c7-9537-567569cc9fcc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:47.611728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630dcf7d-d4c2-45c7-9537-567569cc9fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:47.613106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630dcf7d-d4c2-45c7-9537-567569cc9fcc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:47.618685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 457/890] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:13:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:47.623982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dcf7d-d4c2-45c7-9537-567569cc9fcc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:47.634200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06a66d-3f2f-4807-951a-b5000593222a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:47.672391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5802e18-0287-4bf9-bb2c-ae93144f47df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:47.673014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5802e18-0287-4bf9-bb2c-ae93144f47df None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:47.673288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5802e18-0287-4bf9-bb2c-ae93144f47df None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:47.673597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5802e18-0287-4bf9-bb2c-ae93144f47df None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:47.674301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5802e18-0287-4bf9-bb2c-ae93144f47df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:47.674597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5802e18-0287-4bf9-bb2c-ae93144f47df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:47.676069 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5802e18-0287-4bf9-bb2c-ae93144f47df None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:47.693778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 434/891] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:47.694670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5802e18-0287-4bf9-bb2c-ae93144f47df None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:47.731813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:47.752405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:47.757438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dcf7d-d4c2-45c7-9537-567569cc9fcc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:47.795031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245c968-d186-4c7b-8b8a-a347bad04cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:47.796792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245c968-d186-4c7b-8b8a-a347bad04cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:47.797193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245c968-d186-4c7b-8b8a-a347bad04cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:47.797742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245c968-d186-4c7b-8b8a-a347bad04cd7 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:47.798575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245c968-d186-4c7b-8b8a-a347bad04cd7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:47.799077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245c968-d186-4c7b-8b8a-a347bad04cd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:47.800637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245c968-d186-4c7b-8b8a-a347bad04cd7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:47.806526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 458/892] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:13:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:47.812878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245c968-d186-4c7b-8b8a-a347bad04cd7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:47.822053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5802e18-0287-4bf9-bb2c-ae93144f47df None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:47.861943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4296b4-9ee2-437c-9990-bc56d58c52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:47.862570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4296b4-9ee2-437c-9990-bc56d58c52e1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:47.862840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4296b4-9ee2-437c-9990-bc56d58c52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:47.863109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4296b4-9ee2-437c-9990-bc56d58c52e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:47.871084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4296b4-9ee2-437c-9990-bc56d58c52e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:47.872534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4296b4-9ee2-437c-9990-bc56d58c52e1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-6b4296b4-9ee2-437c-9990-bc56d58c52e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:47.885114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:47.886119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:47.887078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:47.887078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:47.887078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:47.887078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:47.887078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:47.887078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:47.887078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:47.887078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:47.887078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:47.887078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:47.891936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 435/893] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 1 05:13:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:47.892840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4296b4-9ee2-437c-9990-bc56d58c52e1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:47.956005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245c968-d186-4c7b-8b8a-a347bad04cd7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:48.006598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:48.009980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4296b4-9ee2-437c-9990-bc56d58c52e1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:48.020735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49104d0-c17d-4383-99f4-3b315dba4f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:48.031342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49104d0-c17d-4383-99f4-3b315dba4f05 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.031342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49104d0-c17d-4383-99f4-3b315dba4f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.031342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49104d0-c17d-4383-99f4-3b315dba4f05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.031342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49104d0-c17d-4383-99f4-3b315dba4f05 None admin] RBAC: Adding query filter params (interface=public, service_id=88840db09a9a430ba26adba91b1b38b8, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:48.031342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49104d0-c17d-4383-99f4-3b315dba4f05 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:48.031342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49104d0-c17d-4383-99f4-3b315dba4f05 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:48.036920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 459/894] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 1 05:13:47 2022] GET /identity/v3/endpoints?service_id=88840db09a9a430ba26adba91b1b38b8&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:48.042435 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49104d0-c17d-4383-99f4-3b315dba4f05 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:48.053496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:48.056050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfcb12-ecf1-4749-8464-ae298cbf4d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:48.056896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfcb12-ecf1-4749-8464-ae298cbf4d31 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.057695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfcb12-ecf1-4749-8464-ae298cbf4d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.058147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfcb12-ecf1-4749-8464-ae298cbf4d31 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.059384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfcb12-ecf1-4749-8464-ae298cbf4d31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:48.059873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfcb12-ecf1-4749-8464-ae298cbf4d31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:48.063482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfcb12-ecf1-4749-8464-ae298cbf4d31 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:48.088209 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 436/895] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 1 05:13:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:48.089178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfcb12-ecf1-4749-8464-ae298cbf4d31 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:48.175299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49104d0-c17d-4383-99f4-3b315dba4f05 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:48.205924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfcb12-ecf1-4749-8464-ae298cbf4d31 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:48.213764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27b46c-1a96-42ec-95f7-dd0b1476a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:48.214460 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27b46c-1a96-42ec-95f7-dd0b1476a97a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.214739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27b46c-1a96-42ec-95f7-dd0b1476a97a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.215008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27b46c-1a96-42ec-95f7-dd0b1476a97a None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.215695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27b46c-1a96-42ec-95f7-dd0b1476a97a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:48.229613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27b46c-1a96-42ec-95f7-dd0b1476a97a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:48.229613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 460/896] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 1 05:13:48 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:13:48.229613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2d56-320e-4311-855c-4ed5081e3585 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.229613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2d56-320e-4311-855c-4ed5081e3585 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.229613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2d56-320e-4311-855c-4ed5081e3585 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.229613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 461/897] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:48.232149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cb614-9945-474c-bbda-b3483d93adc5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.232439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cb614-9945-474c-bbda-b3483d93adc5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.232715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cb614-9945-474c-bbda-b3483d93adc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.255640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8b4cb614-9945-474c-bbda-b3483d93adc5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:48.256862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8b4cb614-9945-474c-bbda-b3483d93adc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:48.258183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19eef9a-17c1-4319-8117-d72cda8718e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:48.259257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19eef9a-17c1-4319-8117-d72cda8718e1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.259824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19eef9a-17c1-4319-8117-d72cda8718e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.261211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19eef9a-17c1-4319-8117-d72cda8718e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.297086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:48.341387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:48.353494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 462/898] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:13:48.353697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19eef9a-17c1-4319-8117-d72cda8718e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:48.353796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a864a3-3d76-44e2-bb35-155bb10a209c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.353796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a864a3-3d76-44e2-bb35-155bb10a209c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.357107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19eef9a-17c1-4319-8117-d72cda8718e1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:48.357208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a864a3-3d76-44e2-bb35-155bb10a209c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.357208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 463/899] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:48.357208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fb9e4-ef6e-4bf3-90ec-824556cb9934 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.357208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fb9e4-ef6e-4bf3-90ec-824556cb9934 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.357208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fb9e4-ef6e-4bf3-90ec-824556cb9934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.379914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-be4fb9e4-ef6e-4bf3-90ec-824556cb9934 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:48.382134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-be4fb9e4-ef6e-4bf3-90ec-824556cb9934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:48.465593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 464/900] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:48.490367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 437/901] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:48 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:13:48.498286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fb9e4-ef6e-4bf3-90ec-824556cb9934 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:48.602060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fb9e4-ef6e-4bf3-90ec-824556cb9934 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:48.625850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19eef9a-17c1-4319-8117-d72cda8718e1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:48.635006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:48.642814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb75c3-be64-457c-bd40-9bb7db4ad4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:48.643465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb75c3-be64-457c-bd40-9bb7db4ad4dd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.643743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb75c3-be64-457c-bd40-9bb7db4ad4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.644085 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb75c3-be64-457c-bd40-9bb7db4ad4dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.657073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb75c3-be64-457c-bd40-9bb7db4ad4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:48.659051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb75c3-be64-457c-bd40-9bb7db4ad4dd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3dbb75c3-be64-457c-bd40-9bb7db4ad4dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:48.679001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:48.680042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:48.680960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:48.680960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:48.680960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:48.680960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 465/902] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:48.687238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb75c3-be64-457c-bd40-9bb7db4ad4dd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:48.700487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:48.755182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19eef9a-17c1-4319-8117-d72cda8718e1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:48.790623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32cc66-b77d-41ab-900f-5394497caf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:48.791235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32cc66-b77d-41ab-900f-5394497caf8b None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.791509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32cc66-b77d-41ab-900f-5394497caf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.791777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32cc66-b77d-41ab-900f-5394497caf8b None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.792422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32cc66-b77d-41ab-900f-5394497caf8b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:48.793797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32cc66-b77d-41ab-900f-5394497caf8b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:48.810596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb75c3-be64-457c-bd40-9bb7db4ad4dd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:48.844268 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 438/903] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:13:48 2022] POST /identity/v3/services => generated 241 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:13:48.845326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32cc66-b77d-41ab-900f-5394497caf8b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:48.859763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b39db8-c6de-4316-b440-96eca54d3b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:48.860836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b39db8-c6de-4316-b440-96eca54d3b52 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:48.861348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b39db8-c6de-4316-b440-96eca54d3b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:48.861925 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b39db8-c6de-4316-b440-96eca54d3b52 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:48.863226 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b39db8-c6de-4316-b440-96eca54d3b52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:48.863782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b39db8-c6de-4316-b440-96eca54d3b52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:48.865404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b39db8-c6de-4316-b440-96eca54d3b52 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:48.882150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 466/904] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:48.893486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b39db8-c6de-4316-b440-96eca54d3b52 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:48.936236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:49.005776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32cc66-b77d-41ab-900f-5394497caf8b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:49.012268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b39db8-c6de-4316-b440-96eca54d3b52 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:49.027835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:49.048580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278dde5-2e62-495e-bde1-0164715156c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:49.049239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278dde5-2e62-495e-bde1-0164715156c7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:49.049547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278dde5-2e62-495e-bde1-0164715156c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:49.049840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278dde5-2e62-495e-bde1-0164715156c7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:49.050418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278dde5-2e62-495e-bde1-0164715156c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:49.051785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278dde5-2e62-495e-bde1-0164715156c7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:49.054192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c6a6e-c116-40c1-af16-f44f9a848d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:49.055050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c6a6e-c116-40c1-af16-f44f9a848d70 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:49.055494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c6a6e-c116-40c1-af16-f44f9a848d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:49.055939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c6a6e-c116-40c1-af16-f44f9a848d70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8278dde5-2e62-495e-bde1-0164715156c7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:49.064263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:49.065390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:13:49.066350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:49.067118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c6a6e-c116-40c1-af16-f44f9a848d70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:49.067303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c6a6e-c116-40c1-af16-f44f9a848d70 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:49.073494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 439/905] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Sep 1 05:13:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:49.075939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278dde5-2e62-495e-bde1-0164715156c7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-fb6c6a6e-c116-40c1-af16-f44f9a848d70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:49.080908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:49.081990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:49.083492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:49.083492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:49.083492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:49.083492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:49.083492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:49.083492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:49.083492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:49.083492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:49.083492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 467/906] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:49.087479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c6a6e-c116-40c1-af16-f44f9a848d70 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:49.200083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:49.200280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278dde5-2e62-495e-bde1-0164715156c7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:49.204690 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c6a6e-c116-40c1-af16-f44f9a848d70 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:49.244501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbc67f-59d1-48bf-a32b-f3f2336fac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:49.244921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0d491-b8d5-437c-80c2-7f7071cc9d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:49.245114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc67f-59d1-48bf-a32b-f3f2336fac1d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:49.245395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc67f-59d1-48bf-a32b-f3f2336fac1d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:49.245586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0d491-b8d5-437c-80c2-7f7071cc9d1e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:49.245695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc67f-59d1-48bf-a32b-f3f2336fac1d None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:49.245846 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0d491-b8d5-437c-80c2-7f7071cc9d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:49.246106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0d491-b8d5-437c-80c2-7f7071cc9d1e None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:49.246379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbc67f-59d1-48bf-a32b-f3f2336fac1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:49.246683 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbc67f-59d1-48bf-a32b-f3f2336fac1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:49.246789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0d491-b8d5-437c-80c2-7f7071cc9d1e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:49.247090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0d491-b8d5-437c-80c2-7f7071cc9d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:49.248108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbc67f-59d1-48bf-a32b-f3f2336fac1d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:49.248589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0d491-b8d5-437c-80c2-7f7071cc9d1e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:49.255686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 440/907] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:13:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:49.261321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0d491-b8d5-437c-80c2-7f7071cc9d1e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:49.273003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 468/908] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:49.278449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbc67f-59d1-48bf-a32b-f3f2336fac1d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:49.330968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:49.400460 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbc67f-59d1-48bf-a32b-f3f2336fac1d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:49.405277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0d491-b8d5-437c-80c2-7f7071cc9d1e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:49.437704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6c561-7970-4733-9cb4-1e185b920f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:49.438374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6c561-7970-4733-9cb4-1e185b920f01 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:49.438656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6c561-7970-4733-9cb4-1e185b920f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:49.438927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6c561-7970-4733-9cb4-1e185b920f01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:49.439705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6c561-7970-4733-9cb4-1e185b920f01 None admin] RBAC: Adding query filter params (role.id=1830bfc851a04ab3a87c51c2a2c923fb, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=f58aa170e3004197a15dc2dc868d8681) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:49.440016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6c561-7970-4733-9cb4-1e185b920f01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:49.441434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6c561-7970-4733-9cb4-1e185b920f01 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:49.450207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea684e7-177f-4dcb-9bb9-833e0b7b52df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:49.450847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea684e7-177f-4dcb-9bb9-833e0b7b52df None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:49.451117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea684e7-177f-4dcb-9bb9-833e0b7b52df None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:49.451394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea684e7-177f-4dcb-9bb9-833e0b7b52df None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:49.452704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea684e7-177f-4dcb-9bb9-833e0b7b52df None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:49.453013 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea684e7-177f-4dcb-9bb9-833e0b7b52df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:49.453876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 469/909] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:49 2022] GET /identity/v3/role_assignments?user.id=f58aa170e3004197a15dc2dc868d8681&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=1830bfc851a04ab3a87c51c2a2c923fb => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:49.454657 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea684e7-177f-4dcb-9bb9-833e0b7b52df None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:49.460957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 441/910] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:13:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:49.467250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6c561-7970-4733-9cb4-1e185b920f01 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:49.496553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:49.588027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6c561-7970-4733-9cb4-1e185b920f01 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:49.625621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1eed8d-9118-4e8e-9dca-ab2fed81a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:49.626241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eed8d-9118-4e8e-9dca-ab2fed81a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:49.626523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eed8d-9118-4e8e-9dca-ab2fed81a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:49.626846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eed8d-9118-4e8e-9dca-ab2fed81a3b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:49.627558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1eed8d-9118-4e8e-9dca-ab2fed81a3b8 None admin] RBAC: Adding query filter params (interface=public, service_id=e7939a47fdd74a3798f112ebec6f6ffb, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:49.627869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1eed8d-9118-4e8e-9dca-ab2fed81a3b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:49.629242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1eed8d-9118-4e8e-9dca-ab2fed81a3b8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:49.634979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 470/911] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 1 05:13:49 2022] GET /identity/v3/endpoints?service_id=e7939a47fdd74a3798f112ebec6f6ffb&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:49.641405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea684e7-177f-4dcb-9bb9-833e0b7b52df None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:49.751205 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea684e7-177f-4dcb-9bb9-833e0b7b52df None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:49.762504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:49.787874 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab0a90-aef3-418b-a539-6c19ff276944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:49.788474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab0a90-aef3-418b-a539-6c19ff276944 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:49.788748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab0a90-aef3-418b-a539-6c19ff276944 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:49.789017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab0a90-aef3-418b-a539-6c19ff276944 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:49.789647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab0a90-aef3-418b-a539-6c19ff276944 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:49.791008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab0a90-aef3-418b-a539-6c19ff276944 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:49.801959 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 442/912] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 1 05:13:49 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:13:50.204687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b2629-9eb1-4b67-824c-4332c6aa3559 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:50.206832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b2629-9eb1-4b67-824c-4332c6aa3559 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:50.207363 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b2629-9eb1-4b67-824c-4332c6aa3559 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:50.209341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 471/913] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:13:50.214845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbc3bb-2bfb-4cad-895c-0a7ef4772275 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:50.215373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbc3bb-2bfb-4cad-895c-0a7ef4772275 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:50.215845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbc3bb-2bfb-4cad-895c-0a7ef4772275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:50.235430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-50cbc3bb-2bfb-4cad-895c-0a7ef4772275 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:50.236941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-50cbc3bb-2bfb-4cad-895c-0a7ef4772275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:50.308717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 443/914] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:50.580895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b2629-9eb1-4b67-824c-4332c6aa3559 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:50.674269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:50.692298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b2629-9eb1-4b67-824c-4332c6aa3559 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:50.741434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7f850-361c-49b2-8fc2-34a30a9a7c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:50.742395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7f850-361c-49b2-8fc2-34a30a9a7c41 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:50.742872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7f850-361c-49b2-8fc2-34a30a9a7c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:50.743339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7f850-361c-49b2-8fc2-34a30a9a7c41 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:50.744143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7f850-361c-49b2-8fc2-34a30a9a7c41 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:50.745724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7f850-361c-49b2-8fc2-34a30a9a7c41 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-43a7f850-361c-49b2-8fc2-34a30a9a7c41 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:50.755295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:50.756279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:13:50.757125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:50.759825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 472/915] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:13:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:50.766111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbc3bb-2bfb-4cad-895c-0a7ef4772275 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:50.870738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbc3bb-2bfb-4cad-895c-0a7ef4772275 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:50.873234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:50.908445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57f246-ecf1-4582-9477-85711f7742b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:50.909073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f246-ecf1-4582-9477-85711f7742b5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:50.909352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f246-ecf1-4582-9477-85711f7742b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:50.909656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f246-ecf1-4582-9477-85711f7742b5 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:50.910305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57f246-ecf1-4582-9477-85711f7742b5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:50.910603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57f246-ecf1-4582-9477-85711f7742b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:50.912014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57f246-ecf1-4582-9477-85711f7742b5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:50.917762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 444/916] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:13:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:50.923667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7f850-361c-49b2-8fc2-34a30a9a7c41 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:51.007323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160c3cb-3034-4ed9-88e1-338f9dce7448 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.007652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160c3cb-3034-4ed9-88e1-338f9dce7448 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.007902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160c3cb-3034-4ed9-88e1-338f9dce7448 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.010494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 445/917] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:13:51.015377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb7e31-37a5-4d59-a0f5-eba4be56f9d7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.015812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb7e31-37a5-4d59-a0f5-eba4be56f9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.016094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb7e31-37a5-4d59-a0f5-eba4be56f9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.034864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c8cb7e31-37a5-4d59-a0f5-eba4be56f9d7 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:51.035498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7f850-361c-49b2-8fc2-34a30a9a7c41 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:51.036034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c8cb7e31-37a5-4d59-a0f5-eba4be56f9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:51.079669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f8770-ded7-433b-b279-4fd756848d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:51.079669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f8770-ded7-433b-b279-4fd756848d7b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.079669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f8770-ded7-433b-b279-4fd756848d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.079669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f8770-ded7-433b-b279-4fd756848d7b None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.080093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f8770-ded7-433b-b279-4fd756848d7b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:51.082496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f8770-ded7-433b-b279-4fd756848d7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:51.082496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f8770-ded7-433b-b279-4fd756848d7b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:51.093873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 473/918] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 1 05:13:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:51.098397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f8770-ded7-433b-b279-4fd756848d7b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:51.117055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:51.135684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 446/919] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:51.210661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f8770-ded7-433b-b279-4fd756848d7b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:51.246946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeff72-3e79-4d09-8698-2af9e2684125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:51.247942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeff72-3e79-4d09-8698-2af9e2684125 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.248425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeff72-3e79-4d09-8698-2af9e2684125 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.249481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeff72-3e79-4d09-8698-2af9e2684125 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.250377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeeff72-3e79-4d09-8698-2af9e2684125 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:51.251943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeeff72-3e79-4d09-8698-2af9e2684125 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:51.323458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 474/920] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:13:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:51.412227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65ee30-bde3-450c-a2b9-d1000cd1dc1e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.412481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65ee30-bde3-450c-a2b9-d1000cd1dc1e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.412763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65ee30-bde3-450c-a2b9-d1000cd1dc1e None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.414510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 447/921] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:51.414809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeff72-3e79-4d09-8698-2af9e2684125 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:51.422082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116c1b1-3642-428b-9e42-609016e863e1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.422420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116c1b1-3642-428b-9e42-609016e863e1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.422675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116c1b1-3642-428b-9e42-609016e863e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.448623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8116c1b1-3642-428b-9e42-609016e863e1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:51.450743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8116c1b1-3642-428b-9e42-609016e863e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:51.503725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:51.552634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeff72-3e79-4d09-8698-2af9e2684125 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:51.554828 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 448/922] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:51.593396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcafa3e-34b3-4433-a626-93ec1ee6bd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:51.594094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcafa3e-34b3-4433-a626-93ec1ee6bd3e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.594377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcafa3e-34b3-4433-a626-93ec1ee6bd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.594652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcafa3e-34b3-4433-a626-93ec1ee6bd3e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.595246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcafa3e-34b3-4433-a626-93ec1ee6bd3e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:51.596610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcafa3e-34b3-4433-a626-93ec1ee6bd3e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-2fcafa3e-34b3-4433-a626-93ec1ee6bd3e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:51.605794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:51.606788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:51.607663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 475/923] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:13:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:51.614346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116c1b1-3642-428b-9e42-609016e863e1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:51.642502 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:51.728328 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116c1b1-3642-428b-9e42-609016e863e1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:51.765213 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ba947-f3a3-4bec-9649-f286310466af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:51.766029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ba947-f3a3-4bec-9649-f286310466af None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.766350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ba947-f3a3-4bec-9649-f286310466af None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.766878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ba947-f3a3-4bec-9649-f286310466af None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.767252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ba947-f3a3-4bec-9649-f286310466af None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:51.767556 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ba947-f3a3-4bec-9649-f286310466af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:51.768932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ba947-f3a3-4bec-9649-f286310466af None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:51.772064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6f0d6-3ece-4c78-80cb-61a8a934cb5e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.772350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6f0d6-3ece-4c78-80cb-61a8a934cb5e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.772625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6f0d6-3ece-4c78-80cb-61a8a934cb5e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.774634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 449/925] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 1 05:13:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:51.777942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 476/925] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:51.779439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784afc9-f354-4295-8f44-5a6ce0c0b2dd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.779750 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784afc9-f354-4295-8f44-5a6ce0c0b2dd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.780050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784afc9-f354-4295-8f44-5a6ce0c0b2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.781133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6f0d6-3ece-4c78-80cb-61a8a934cb5e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:51.798635 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2784afc9-f354-4295-8f44-5a6ce0c0b2dd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:51.799875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2784afc9-f354-4295-8f44-5a6ce0c0b2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:51.879605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 450/926] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:51.880577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784afc9-f354-4295-8f44-5a6ce0c0b2dd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:51.904928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:51.908905 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6f0d6-3ece-4c78-80cb-61a8a934cb5e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:51.947400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:51.955908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fae9ec-0b69-43a1-b1fe-3bbd943b2f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:51.956734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae9ec-0b69-43a1-b1fe-3bbd943b2f77 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:51.957068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae9ec-0b69-43a1-b1fe-3bbd943b2f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:51.957350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae9ec-0b69-43a1-b1fe-3bbd943b2f77 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:51.958334 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fae9ec-0b69-43a1-b1fe-3bbd943b2f77 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:51.958657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fae9ec-0b69-43a1-b1fe-3bbd943b2f77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:51.960324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fae9ec-0b69-43a1-b1fe-3bbd943b2f77 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:51.969602 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 477/927] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 1 05:13:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:51.974132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fae9ec-0b69-43a1-b1fe-3bbd943b2f77 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.011056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784afc9-f354-4295-8f44-5a6ce0c0b2dd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:52.051755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98b8c0-6c94-4419-9759-4670879cfd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:52.052861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98b8c0-6c94-4419-9759-4670879cfd9b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:52.053347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98b8c0-6c94-4419-9759-4670879cfd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:52.053850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98b8c0-6c94-4419-9759-4670879cfd9b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:52.065982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98b8c0-6c94-4419-9759-4670879cfd9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8a98b8c0-6c94-4419-9759-4670879cfd9b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:52.071573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:52.072780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:13:52.073674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:52.081071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 451/928] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 1 05:13:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:52.082953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98b8c0-6c94-4419-9759-4670879cfd9b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.095527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fae9ec-0b69-43a1-b1fe-3bbd943b2f77 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:52.137931 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02875f-7e28-4d00-8a09-1b9277cc2bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:52.138832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02875f-7e28-4d00-8a09-1b9277cc2bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:52.139182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02875f-7e28-4d00-8a09-1b9277cc2bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:52.139481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02875f-7e28-4d00-8a09-1b9277cc2bc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:52.140194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02875f-7e28-4d00-8a09-1b9277cc2bc5 None admin] RBAC: Adding query filter params (interface=public, service_id=a56bda04020c41f6b6f3436a57619f97, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:52.140505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02875f-7e28-4d00-8a09-1b9277cc2bc5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:52.142335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02875f-7e28-4d00-8a09-1b9277cc2bc5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:52.148572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 478/929] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 1 05:13:51 2022] GET /identity/v3/endpoints?service_id=a56bda04020c41f6b6f3436a57619f97&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:52.155325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02875f-7e28-4d00-8a09-1b9277cc2bc5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.179617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:52.201911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98b8c0-6c94-4419-9759-4670879cfd9b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:52.221195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:52.238889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835217b-f592-4cb5-bb66-31e236305609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:52.239531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835217b-f592-4cb5-bb66-31e236305609 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:52.239800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835217b-f592-4cb5-bb66-31e236305609 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:52.240066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835217b-f592-4cb5-bb66-31e236305609 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:52.244209 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835217b-f592-4cb5-bb66-31e236305609 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:52.244209 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835217b-f592-4cb5-bb66-31e236305609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:52.244209 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835217b-f592-4cb5-bb66-31e236305609 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:52.255935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 452/930] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 1 05:13:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:52.255935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835217b-f592-4cb5-bb66-31e236305609 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.273254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02875f-7e28-4d00-8a09-1b9277cc2bc5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:52.308948 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1c812-8a26-45a0-872c-c7034cf413ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:52.309597 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c812-8a26-45a0-872c-c7034cf413ae None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:52.309879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c812-8a26-45a0-872c-c7034cf413ae None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:52.310151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c812-8a26-45a0-872c-c7034cf413ae None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:52.310758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1c812-8a26-45a0-872c-c7034cf413ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:52.312851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1c812-8a26-45a0-872c-c7034cf413ae None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:52.318797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 479/931] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 1 05:13:52 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:13:52.319571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1c812-8a26-45a0-872c-c7034cf413ae None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.378468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835217b-f592-4cb5-bb66-31e236305609 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:52.416989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf782dc3-f8ba-4626-b7d1-5a00dcde81d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:52.417665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf782dc3-f8ba-4626-b7d1-5a00dcde81d4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:52.417947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf782dc3-f8ba-4626-b7d1-5a00dcde81d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:52.418219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf782dc3-f8ba-4626-b7d1-5a00dcde81d4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:52.418816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf782dc3-f8ba-4626-b7d1-5a00dcde81d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:52.420222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf782dc3-f8ba-4626-b7d1-5a00dcde81d4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-bf782dc3-f8ba-4626-b7d1-5a00dcde81d4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:52.431033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:52.432089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:13:52.433009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:52.437007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 453/932] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Sep 1 05:13:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:52.438433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf782dc3-f8ba-4626-b7d1-5a00dcde81d4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.448592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1c812-8a26-45a0-872c-c7034cf413ae None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:52.465438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:52.476881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:52.495234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fcd876-eb37-4b7b-9d06-c443da633bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:52.495988 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcd876-eb37-4b7b-9d06-c443da633bcb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:52.496438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcd876-eb37-4b7b-9d06-c443da633bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:52.496733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fcd876-eb37-4b7b-9d06-c443da633bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:52.510514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fcd876-eb37-4b7b-9d06-c443da633bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:52.511977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fcd876-eb37-4b7b-9d06-c443da633bcb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-54fcd876-eb37-4b7b-9d06-c443da633bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:52.533933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:52.534967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:52.536562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:52.536562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:52.536562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:52.536562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 480/933] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:52.541660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fcd876-eb37-4b7b-9d06-c443da633bcb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.588263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf782dc3-f8ba-4626-b7d1-5a00dcde81d4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:52.637114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ae0e8-21d2-4c1d-8a50-4a21dbc67403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:52.638259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ae0e8-21d2-4c1d-8a50-4a21dbc67403 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:52.639662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ae0e8-21d2-4c1d-8a50-4a21dbc67403 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:52.640156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ae0e8-21d2-4c1d-8a50-4a21dbc67403 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:52.641899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ae0e8-21d2-4c1d-8a50-4a21dbc67403 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:52.642442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ae0e8-21d2-4c1d-8a50-4a21dbc67403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:52.644060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ae0e8-21d2-4c1d-8a50-4a21dbc67403 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:52.656616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 454/934] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:13:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:52.658262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ae0e8-21d2-4c1d-8a50-4a21dbc67403 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.682726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fcd876-eb37-4b7b-9d06-c443da633bcb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:52.726031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab2bb1-5a88-4df6-8c0e-6c92971500bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:52.726642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab2bb1-5a88-4df6-8c0e-6c92971500bb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:52.726918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab2bb1-5a88-4df6-8c0e-6c92971500bb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:52.727189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab2bb1-5a88-4df6-8c0e-6c92971500bb None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:52.727830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab2bb1-5a88-4df6-8c0e-6c92971500bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:52.728130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab2bb1-5a88-4df6-8c0e-6c92971500bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:52.729950 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab2bb1-5a88-4df6-8c0e-6c92971500bb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:52.747927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:52.751999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 481/935] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:52.757966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab2bb1-5a88-4df6-8c0e-6c92971500bb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.784975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ae0e8-21d2-4c1d-8a50-4a21dbc67403 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:52.797040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:52.827072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b5aaf-9bfc-4ba8-b3a1-348312119b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:52.827786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b5aaf-9bfc-4ba8-b3a1-348312119b58 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:52.828415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b5aaf-9bfc-4ba8-b3a1-348312119b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:52.828764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b5aaf-9bfc-4ba8-b3a1-348312119b58 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:52.829489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b5aaf-9bfc-4ba8-b3a1-348312119b58 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:52.830232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b5aaf-9bfc-4ba8-b3a1-348312119b58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:52.832502 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b5aaf-9bfc-4ba8-b3a1-348312119b58 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:52.840414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 455/936] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:13:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:52.846382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b5aaf-9bfc-4ba8-b3a1-348312119b58 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.887580 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab2bb1-5a88-4df6-8c0e-6c92971500bb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:52.926888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42270d-0fa9-4eac-9886-6042ccf4e84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:52.927621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42270d-0fa9-4eac-9886-6042ccf4e84d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:52.927975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42270d-0fa9-4eac-9886-6042ccf4e84d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:52.928326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42270d-0fa9-4eac-9886-6042ccf4e84d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:52.936917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42270d-0fa9-4eac-9886-6042ccf4e84d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:52.938550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42270d-0fa9-4eac-9886-6042ccf4e84d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ec42270d-0fa9-4eac-9886-6042ccf4e84d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:52.951099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:52.952468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:52.953418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 482/937] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 1 05:13:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:52.958044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42270d-0fa9-4eac-9886-6042ccf4e84d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:52.964504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b5aaf-9bfc-4ba8-b3a1-348312119b58 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:53.004797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:53.010938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5d300-52dc-4809-968f-fbd02190fe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:53.011699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5d300-52dc-4809-968f-fbd02190fe2f None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:53.011980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5d300-52dc-4809-968f-fbd02190fe2f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:53.012259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5d300-52dc-4809-968f-fbd02190fe2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:53.012941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5d300-52dc-4809-968f-fbd02190fe2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:53.015206 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5d300-52dc-4809-968f-fbd02190fe2f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:53.070470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:53.078341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42270d-0fa9-4eac-9886-6042ccf4e84d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:53.117708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 456/938] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:13:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:13:53.128586 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7dda7-c492-4fd3-abf9-44dae174d5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:53.129261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7dda7-c492-4fd3-abf9-44dae174d5ba None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:53.129579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7dda7-c492-4fd3-abf9-44dae174d5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:53.129954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7dda7-c492-4fd3-abf9-44dae174d5ba None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:53.130710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7dda7-c492-4fd3-abf9-44dae174d5ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:53.131068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7dda7-c492-4fd3-abf9-44dae174d5ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:53.133010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7dda7-c492-4fd3-abf9-44dae174d5ba None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:53.156999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 483/939] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 1 05:13:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:53.162699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5d300-52dc-4809-968f-fbd02190fe2f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:53.290861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5d300-52dc-4809-968f-fbd02190fe2f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:53.329755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10895f4-cc9c-4851-ad02-220d39522e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:53.330664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10895f4-cc9c-4851-ad02-220d39522e27 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:53.331114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10895f4-cc9c-4851-ad02-220d39522e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:53.331567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10895f4-cc9c-4851-ad02-220d39522e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:53.343766 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10895f4-cc9c-4851-ad02-220d39522e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:53.345654 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10895f4-cc9c-4851-ad02-220d39522e27 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-f10895f4-cc9c-4851-ad02-220d39522e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:53.362160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:53.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:53.364086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:53.364086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:53.364086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:53.366042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 457/940] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:53.373478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7dda7-c492-4fd3-abf9-44dae174d5ba None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:53.390316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c3eff-ffd9-4fcc-bbb1-bd4f0c9a5d17 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:53.390744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c3eff-ffd9-4fcc-bbb1-bd4f0c9a5d17 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:53.391118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c3eff-ffd9-4fcc-bbb1-bd4f0c9a5d17 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:53.392745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 458/941] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:13:53.398464 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90bf8a-79e2-47ac-8786-b37a6d9d1bad None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:53.398866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90bf8a-79e2-47ac-8786-b37a6d9d1bad None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:53.399242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90bf8a-79e2-47ac-8786-b37a6d9d1bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:53.422801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6d90bf8a-79e2-47ac-8786-b37a6d9d1bad None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:53.424186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6d90bf8a-79e2-47ac-8786-b37a6d9d1bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:53.472219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:53.517482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 459/942] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:13:53.520650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7dda7-c492-4fd3-abf9-44dae174d5ba None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:53.557825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfa8c6-99db-4f16-92d8-8d4039c0a4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:53.558584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa8c6-99db-4f16-92d8-8d4039c0a4cf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:53.558967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa8c6-99db-4f16-92d8-8d4039c0a4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:53.559340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa8c6-99db-4f16-92d8-8d4039c0a4cf None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:53.560084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfa8c6-99db-4f16-92d8-8d4039c0a4cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:53.560516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfa8c6-99db-4f16-92d8-8d4039c0a4cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:53.562570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfa8c6-99db-4f16-92d8-8d4039c0a4cf None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:53.574929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 484/943] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:53.581171 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90bf8a-79e2-47ac-8786-b37a6d9d1bad None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:53.687957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90bf8a-79e2-47ac-8786-b37a6d9d1bad None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:53.731390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae123e1e-89c2-4ead-b3f6-bba5625a5ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:53.732273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae123e1e-89c2-4ead-b3f6-bba5625a5ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:53.732703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae123e1e-89c2-4ead-b3f6-bba5625a5ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:53.733906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae123e1e-89c2-4ead-b3f6-bba5625a5ef4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:53.741999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae123e1e-89c2-4ead-b3f6-bba5625a5ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:53.743754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae123e1e-89c2-4ead-b3f6-bba5625a5ef4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-ae123e1e-89c2-4ead-b3f6-bba5625a5ef4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:53.755520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:53.756568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:53.757702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:53.757702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:53.757702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:53.757702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:53.757702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:53.757702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:53.757702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:53.757702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:53.759758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 460/944] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:53.769507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfa8c6-99db-4f16-92d8-8d4039c0a4cf None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:53.777411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:53.818139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae123e1e-89c2-4ead-b3f6-bba5625a5ef4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:53.853922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:53.891789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfa8c6-99db-4f16-92d8-8d4039c0a4cf None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:53.929132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec2d33-bf5b-454c-9a7d-ff10c3a0fd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:53.930234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec2d33-bf5b-454c-9a7d-ff10c3a0fd72 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:53.930763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec2d33-bf5b-454c-9a7d-ff10c3a0fd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:53.931248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec2d33-bf5b-454c-9a7d-ff10c3a0fd72 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:53.932182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec2d33-bf5b-454c-9a7d-ff10c3a0fd72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:53.932702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec2d33-bf5b-454c-9a7d-ff10c3a0fd72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:53.934343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec2d33-bf5b-454c-9a7d-ff10c3a0fd72 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:53.940735 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae123e1e-89c2-4ead-b3f6-bba5625a5ef4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:53.949698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 485/945] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:53.955449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec2d33-bf5b-454c-9a7d-ff10c3a0fd72 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:53.992085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:53.993605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:53.993897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:53.994175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:53.994847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:53.996292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:53.999511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: WARNING keystone.server.flask.common [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 05:13:54.044989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:54.054825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.sql.core [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 05:13:54.054825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 05:13:54.054825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [parameters: {'id': '45a0871865754498ba404cf20fa9a3d7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 05:13:54.054825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81577) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 45a0871865754498ba404cf20fa9a3d7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 45a0871865754498ba404cf20fa9a3d7. Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:13:54.080135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result.read() Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 05:13:54.081049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application yield resource Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise self.value Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application session.commit() Sep 01 05:13:54.082070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.session.flush() Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application compat.raise_( Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 05:13:54.083021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 05:13:54.083930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result.read() Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:13:54.084736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [parameters: {'id': '45a0871865754498ba404cf20fa9a3d7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:54.087125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:54.089904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 45a0871865754498ba404cf20fa9a3d7. Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 461/946] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Sep 1 05:13:53 2022] POST /identity/v3/projects => generated 262 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:54.092434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:54.126548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec2d33-bf5b-454c-9a7d-ff10c3a0fd72 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:54.198408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737b4af-0408-4d69-9358-a9ceab5bae4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:54.199777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737b4af-0408-4d69-9358-a9ceab5bae4c None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:54.199777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737b4af-0408-4d69-9358-a9ceab5bae4c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:54.199947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737b4af-0408-4d69-9358-a9ceab5bae4c None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/f58aa170e3004197a15dc2dc868d8681/roles/1830bfc851a04ab3a87c51c2a2c923fb` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:54.210321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737b4af-0408-4d69-9358-a9ceab5bae4c None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=f58aa170e3004197a15dc2dc868d8681, role_id=1830bfc851a04ab3a87c51c2a2c923fb)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:54.211873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737b4af-0408-4d69-9358-a9ceab5bae4c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:54.259465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:54.276094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79bd3a-b80a-4344-a434-b8ae252ad590 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:54.301889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 486/947] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:13:53 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/f58aa170e3004197a15dc2dc868d8681/roles/1830bfc851a04ab3a87c51c2a2c923fb => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:13:54.343584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99916a-37b9-4e8b-8ea3-89a9a0295c4f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:54.343912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99916a-37b9-4e8b-8ea3-89a9a0295c4f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:54.344189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99916a-37b9-4e8b-8ea3-89a9a0295c4f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:54.346167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fe5d6-161d-43c2-a406-7324af7affc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:54.347909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fe5d6-161d-43c2-a406-7324af7affc3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:54.350276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 487/948] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:13:54.350916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fe5d6-161d-43c2-a406-7324af7affc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:54.351473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fe5d6-161d-43c2-a406-7324af7affc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:54.351704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab76e94-3c48-4326-abd3-459ff4d8649f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:54.351992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab76e94-3c48-4326-abd3-459ff4d8649f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:54.352268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab76e94-3c48-4326-abd3-459ff4d8649f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:54.371719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fe5d6-161d-43c2-a406-7324af7affc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:54.376255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fe5d6-161d-43c2-a406-7324af7affc3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:54.376919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0ab76e94-3c48-4326-abd3-459ff4d8649f None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:54.379649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0ab76e94-3c48-4326-abd3-459ff4d8649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-730fe5d6-161d-43c2-a406-7324af7affc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:54.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:54.402516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:54.403419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:54.403419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:54.403419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:54.403419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:54.403419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:54.403419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:54.403419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:54.403419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:54.403419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 462/949] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:54.413503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fe5d6-161d-43c2-a406-7324af7affc3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:54.529494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 488/950] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:54.556959 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fe5d6-161d-43c2-a406-7324af7affc3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:54.593260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fdf39-9c91-4229-ad7f-6ac89d574512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:54.593994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fdf39-9c91-4229-ad7f-6ac89d574512 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:54.594272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fdf39-9c91-4229-ad7f-6ac89d574512 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:54.594541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fdf39-9c91-4229-ad7f-6ac89d574512 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:54.595202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fdf39-9c91-4229-ad7f-6ac89d574512 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:54.595503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fdf39-9c91-4229-ad7f-6ac89d574512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:54.596928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fdf39-9c91-4229-ad7f-6ac89d574512 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:54.620089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 463/951] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:13:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:54.816313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab76e94-3c48-4326-abd3-459ff4d8649f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:54.881584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:54.924919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab76e94-3c48-4326-abd3-459ff4d8649f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:54.962060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e626c-6a0d-4d50-be54-f5345ac91061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:54.962709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e626c-6a0d-4d50-be54-f5345ac91061 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:54.963039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e626c-6a0d-4d50-be54-f5345ac91061 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:54.963263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e626c-6a0d-4d50-be54-f5345ac91061 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:54.963857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e626c-6a0d-4d50-be54-f5345ac91061 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:54.965223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e626c-6a0d-4d50-be54-f5345ac91061 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-663e626c-6a0d-4d50-be54-f5345ac91061 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:54.974702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:54.975870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:54.976805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:54.979408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 489/952] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:13:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:54.982563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fdf39-9c91-4229-ad7f-6ac89d574512 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:54.992033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:55.091734 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fdf39-9c91-4229-ad7f-6ac89d574512 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:55.127569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821e6bf-8e76-43b3-b27c-ab6dec5bc25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:55.128176 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821e6bf-8e76-43b3-b27c-ab6dec5bc25d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:55.128450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821e6bf-8e76-43b3-b27c-ab6dec5bc25d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:55.128717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821e6bf-8e76-43b3-b27c-ab6dec5bc25d None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:55.129340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821e6bf-8e76-43b3-b27c-ab6dec5bc25d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:55.129670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821e6bf-8e76-43b3-b27c-ab6dec5bc25d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:55.131049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821e6bf-8e76-43b3-b27c-ab6dec5bc25d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:55.136364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 464/953] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 1 05:13:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:55.142817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e626c-6a0d-4d50-be54-f5345ac91061 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:55.216389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ec942-bb4a-4333-a2d5-0cbfb6fc4396 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:55.216596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ec942-bb4a-4333-a2d5-0cbfb6fc4396 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:55.216849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ec942-bb4a-4333-a2d5-0cbfb6fc4396 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:55.219415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 465/954] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:55.224355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7f9c1-7185-4a52-8c02-551005866f1c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:55.224636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7f9c1-7185-4a52-8c02-551005866f1c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:55.224908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7f9c1-7185-4a52-8c02-551005866f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:55.252911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e8c7f9c1-7185-4a52-8c02-551005866f1c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:55.252911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e8c7f9c1-7185-4a52-8c02-551005866f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:55.261762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e626c-6a0d-4d50-be54-f5345ac91061 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:55.288760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:55.302472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee403a-37d5-4c41-ba2a-8f96fc3c6162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:55.303081 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee403a-37d5-4c41-ba2a-8f96fc3c6162 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:55.303455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee403a-37d5-4c41-ba2a-8f96fc3c6162 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:55.303657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee403a-37d5-4c41-ba2a-8f96fc3c6162 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:55.304791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee403a-37d5-4c41-ba2a-8f96fc3c6162 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:55.305095 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee403a-37d5-4c41-ba2a-8f96fc3c6162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:55.307089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee403a-37d5-4c41-ba2a-8f96fc3c6162 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:55.313639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 490/955] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 1 05:13:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:55.319070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee403a-37d5-4c41-ba2a-8f96fc3c6162 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:55.337700 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:55.349580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 466/956] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:13:55.430763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee403a-37d5-4c41-ba2a-8f96fc3c6162 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:55.468874 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630cbc9-7214-445b-824e-913ca3579559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:55.469505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630cbc9-7214-445b-824e-913ca3579559 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:55.469788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630cbc9-7214-445b-824e-913ca3579559 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:55.470058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630cbc9-7214-445b-824e-913ca3579559 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:55.470728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630cbc9-7214-445b-824e-913ca3579559 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:55.472075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630cbc9-7214-445b-824e-913ca3579559 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:55.534508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 491/957] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:13:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:55.632743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7f9c1-7185-4a52-8c02-551005866f1c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:55.752012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7f9c1-7185-4a52-8c02-551005866f1c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:55.794799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb3b41-b398-4f87-85a9-6eab93e02797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:55.795924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3b41-b398-4f87-85a9-6eab93e02797 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:55.796199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3b41-b398-4f87-85a9-6eab93e02797 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:55.796463 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3b41-b398-4f87-85a9-6eab93e02797 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:55.797217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb3b41-b398-4f87-85a9-6eab93e02797 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:55.798616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb3b41-b398-4f87-85a9-6eab93e02797 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-b6cb3b41-b398-4f87-85a9-6eab93e02797 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:55.807658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:55.808831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:55.809786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 467/958] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:55.815810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630cbc9-7214-445b-824e-913ca3579559 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:55.867315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:55.930205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630cbc9-7214-445b-824e-913ca3579559 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:55.967945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bca6ef-a272-4df0-bd9e-199ec9adb930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:55.968566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bca6ef-a272-4df0-bd9e-199ec9adb930 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:55.968845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bca6ef-a272-4df0-bd9e-199ec9adb930 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:55.969119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bca6ef-a272-4df0-bd9e-199ec9adb930 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:55.969834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bca6ef-a272-4df0-bd9e-199ec9adb930 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:55.970151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bca6ef-a272-4df0-bd9e-199ec9adb930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:55.971504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bca6ef-a272-4df0-bd9e-199ec9adb930 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:55.976886 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 492/959] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:13:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:55.982317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb3b41-b398-4f87-85a9-6eab93e02797 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:56.052987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:56.100582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb3b41-b398-4f87-85a9-6eab93e02797 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:56.144093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922f38b-6ac1-4af7-bee5-4057d741bfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:56.144705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922f38b-6ac1-4af7-bee5-4057d741bfc1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:56.144978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922f38b-6ac1-4af7-bee5-4057d741bfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:56.145245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922f38b-6ac1-4af7-bee5-4057d741bfc1 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:56.145906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922f38b-6ac1-4af7-bee5-4057d741bfc1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:56.146207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922f38b-6ac1-4af7-bee5-4057d741bfc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:56.147604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922f38b-6ac1-4af7-bee5-4057d741bfc1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:56.152679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 468/960] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:13:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:56.423858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdea297-1cb0-467d-b8f8-66ded1e9b61b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:56.424219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdea297-1cb0-467d-b8f8-66ded1e9b61b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:56.424421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdea297-1cb0-467d-b8f8-66ded1e9b61b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:56.426079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 493/961] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:56.431280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0b200-1031-460b-a292-2d02dbfc1caa None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:56.431570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0b200-1031-460b-a292-2d02dbfc1caa None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:56.431844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0b200-1031-460b-a292-2d02dbfc1caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:56.450777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c7a0b200-1031-460b-a292-2d02dbfc1caa None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:56.452518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c7a0b200-1031-460b-a292-2d02dbfc1caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:56.524350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 469/962] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:56.746653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1082329-6d27-4673-ba2b-f3d6cd70ecb5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:56.747160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1082329-6d27-4673-ba2b-f3d6cd70ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:56.747268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1082329-6d27-4673-ba2b-f3d6cd70ecb5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:56.749555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 494/963] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:56.754759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1cad6-4731-40e9-aea8-644fdd6c91c8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:56.755052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1cad6-4731-40e9-aea8-644fdd6c91c8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:56.755333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1cad6-4731-40e9-aea8-644fdd6c91c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:56.775075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c1e1cad6-4731-40e9-aea8-644fdd6c91c8 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:56.776287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c1e1cad6-4731-40e9-aea8-644fdd6c91c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:56.797311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1082329-6d27-4673-ba2b-f3d6cd70ecb5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:56.837731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:56.861807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 470/964] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:56.915246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1082329-6d27-4673-ba2b-f3d6cd70ecb5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:56.922841 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:56.951221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab821d-67d7-423b-8717-f87c779cdbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:56.952385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab821d-67d7-423b-8717-f87c779cdbec None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:56.952811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab821d-67d7-423b-8717-f87c779cdbec None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:56.953146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab821d-67d7-423b-8717-f87c779cdbec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:56.964374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab821d-67d7-423b-8717-f87c779cdbec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e7ab821d-67d7-423b-8717-f87c779cdbec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:13:56.969616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:56.970569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:56.971422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 495/965] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 1 05:13:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:56.976939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1cad6-4731-40e9-aea8-644fdd6c91c8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:57.082125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1cad6-4731-40e9-aea8-644fdd6c91c8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:57.117684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941b0bd-83b3-42b8-ba75-76eac068de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:57.118295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941b0bd-83b3-42b8-ba75-76eac068de54 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:57.118574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941b0bd-83b3-42b8-ba75-76eac068de54 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:57.118853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941b0bd-83b3-42b8-ba75-76eac068de54 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:57.119518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941b0bd-83b3-42b8-ba75-76eac068de54 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:57.119825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941b0bd-83b3-42b8-ba75-76eac068de54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:57.121203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941b0bd-83b3-42b8-ba75-76eac068de54 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:57.129944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 471/966] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 1 05:13:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:57.135552 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab821d-67d7-423b-8717-f87c779cdbec None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:57.137335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941b0bd-83b3-42b8-ba75-76eac068de54 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:57.204703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:57.254757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab821d-67d7-423b-8717-f87c779cdbec None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:57.284809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941b0bd-83b3-42b8-ba75-76eac068de54 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:57.291963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a724d5-b062-4e6e-b9d2-9c2edfb80f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:57.292738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a724d5-b062-4e6e-b9d2-9c2edfb80f6e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:57.293117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a724d5-b062-4e6e-b9d2-9c2edfb80f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:57.293507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a724d5-b062-4e6e-b9d2-9c2edfb80f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:57.305848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a724d5-b062-4e6e-b9d2-9c2edfb80f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:57.307499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a724d5-b062-4e6e-b9d2-9c2edfb80f6e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f7a724d5-b062-4e6e-b9d2-9c2edfb80f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:57.325295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:57.326643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:57.327733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:57.327733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:57.327733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:57.329314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 496/967] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:57.337583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a724d5-b062-4e6e-b9d2-9c2edfb80f6e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:57.337735 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66424da9-afe2-4a53-bc9d-d037099cda18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:57.338079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66424da9-afe2-4a53-bc9d-d037099cda18 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:57.338448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66424da9-afe2-4a53-bc9d-d037099cda18 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:57.338854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66424da9-afe2-4a53-bc9d-d037099cda18 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:57.339580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66424da9-afe2-4a53-bc9d-d037099cda18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:57.342434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66424da9-afe2-4a53-bc9d-d037099cda18 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:57.360074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:57.383606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.sql.core [None req-66424da9-afe2-4a53-bc9d-d037099cda18 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 05:13:57.383606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 05:13:57.383606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [parameters: {'user_id': '12d15992816740c4893e533a9c9ad220', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 05:13:57.383606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81577) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-66424da9-afe2-4a53-bc9d-d037099cda18 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result.read() Sep 01 05:13:57.393620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:13:57.394631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application yield resource Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise self.value Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application session.commit() Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 05:13:57.399367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.session.flush() Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application compat.raise_( Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 05:13:57.400392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:13:57.401333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result.read() Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:13:57.402247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [parameters: {'user_id': '12d15992816740c4893e533a9c9ad220', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:57.403488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 05:13:57.406418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 05:13:57.408604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 05:13:57.408604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 05:13:57.408604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 05:13:57.408604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 05:13:57.408604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 05:13:57.408604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:57.408604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 472/968] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 1 05:13:57 2022] POST /identity/v3/users => generated 160 bytes in 258 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:57.410799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66424da9-afe2-4a53-bc9d-d037099cda18 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:57.461403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a724d5-b062-4e6e-b9d2-9c2edfb80f6e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:57.500123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b28945-49a1-4ee0-a1e8-5500a089652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:57.500734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b28945-49a1-4ee0-a1e8-5500a089652a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:57.501015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b28945-49a1-4ee0-a1e8-5500a089652a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:57.501287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b28945-49a1-4ee0-a1e8-5500a089652a None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:57.502040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b28945-49a1-4ee0-a1e8-5500a089652a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:57.502342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b28945-49a1-4ee0-a1e8-5500a089652a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:57.503700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b28945-49a1-4ee0-a1e8-5500a089652a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:57.517342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 497/969] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:57.522323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66424da9-afe2-4a53-bc9d-d037099cda18 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:57.522554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b28945-49a1-4ee0-a1e8-5500a089652a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:57.534287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:57.561832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f2d52-d48a-4afb-88a5-f5a290ea08e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:57.562829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f2d52-d48a-4afb-88a5-f5a290ea08e7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:57.563421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f2d52-d48a-4afb-88a5-f5a290ea08e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:57.563893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f2d52-d48a-4afb-88a5-f5a290ea08e7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:57.576778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f2d52-d48a-4afb-88a5-f5a290ea08e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:57.578492 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f2d52-d48a-4afb-88a5-f5a290ea08e7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-4a4f2d52-d48a-4afb-88a5-f5a290ea08e7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:57.595084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:57.596083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:57.597053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:57.597053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:57.597053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:57.597053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:57.597053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:57.597053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:57.597053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:57.597053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:57.597053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:13:57.597053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:57.601110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 473/970] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:13:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:57.602428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f2d52-d48a-4afb-88a5-f5a290ea08e7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:57.629278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:57.654172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b28945-49a1-4ee0-a1e8-5500a089652a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:57.698162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f8989-26ba-4ebf-9a0c-53363917fef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:57.699170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f8989-26ba-4ebf-9a0c-53363917fef0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:57.699448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f8989-26ba-4ebf-9a0c-53363917fef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:57.699729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f8989-26ba-4ebf-9a0c-53363917fef0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:57.708449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f8989-26ba-4ebf-9a0c-53363917fef0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:57.710058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f8989-26ba-4ebf-9a0c-53363917fef0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-062f8989-26ba-4ebf-9a0c-53363917fef0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:13:57.727089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:57.728115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:57.729964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:57.729964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:57.729964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:13:57.729964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:13:57.729964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:13:57.729964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:13:57.729964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:13:57.729964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:13:57.729964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:13:57.729964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:57.733773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 498/971] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 1 05:13:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:57.735223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60deaf50-6412-4882-8a99-978d9d9b4f84 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:57.735501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60deaf50-6412-4882-8a99-978d9d9b4f84 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:57.735771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60deaf50-6412-4882-8a99-978d9d9b4f84 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:57.737219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 499/972] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:13:57.738461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f2d52-d48a-4afb-88a5-f5a290ea08e7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:57.742657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60deaf50-6412-4882-8a99-978d9d9b4f84 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:57.779482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadeb1ab-c1ac-4c66-8b95-bde91bebef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:57.780262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadeb1ab-c1ac-4c66-8b95-bde91bebef16 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:57.780543 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadeb1ab-c1ac-4c66-8b95-bde91bebef16 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:57.781636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadeb1ab-c1ac-4c66-8b95-bde91bebef16 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:57.782356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadeb1ab-c1ac-4c66-8b95-bde91bebef16 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:57.782673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadeb1ab-c1ac-4c66-8b95-bde91bebef16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:57.784120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadeb1ab-c1ac-4c66-8b95-bde91bebef16 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:57.833487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 474/973] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 1 05:13:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:57.833487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfda05-2346-4d29-99ea-55b4063526be None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:57.833487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfda05-2346-4d29-99ea-55b4063526be None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:57.833487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfda05-2346-4d29-99ea-55b4063526be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:57.849581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-bcbfda05-2346-4d29-99ea-55b4063526be None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:57.851110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bcbfda05-2346-4d29-99ea-55b4063526be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:57.864529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60deaf50-6412-4882-8a99-978d9d9b4f84 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:57.876590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:57.906623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383ff12-54a0-4d10-b6a3-b0afb723d13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:57.911780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383ff12-54a0-4d10-b6a3-b0afb723d13f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:57.911780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383ff12-54a0-4d10-b6a3-b0afb723d13f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:57.911780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383ff12-54a0-4d10-b6a3-b0afb723d13f None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:57.911780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383ff12-54a0-4d10-b6a3-b0afb723d13f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:57.911780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383ff12-54a0-4d10-b6a3-b0afb723d13f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:57.912214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383ff12-54a0-4d10-b6a3-b0afb723d13f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:57.924722 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:57.932965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 475/974] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:13:57.946066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 500/975] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 1 05:13:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:57.956382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfda05-2346-4d29-99ea-55b4063526be None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:58.073111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfda05-2346-4d29-99ea-55b4063526be None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:58.112595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53654e8-a47a-47c4-843b-8e33c7e72880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:58.113185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53654e8-a47a-47c4-843b-8e33c7e72880 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.113414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53654e8-a47a-47c4-843b-8e33c7e72880 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.113748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53654e8-a47a-47c4-843b-8e33c7e72880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.141010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:58.187938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53654e8-a47a-47c4-843b-8e33c7e72880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:58.188550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cfa3e-e3c2-4f83-9e46-d02cf472a7ee None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.188844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cfa3e-e3c2-4f83-9e46-d02cf472a7ee None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.189125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cfa3e-e3c2-4f83-9e46-d02cf472a7ee None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.189391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53654e8-a47a-47c4-843b-8e33c7e72880 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:58.190759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 501/976] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:13:58.196446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5490c04-926f-47cc-be2e-5beef491024d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.196446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5490c04-926f-47cc-be2e-5beef491024d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.196446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5490c04-926f-47cc-be2e-5beef491024d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.214795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f5490c04-926f-47cc-be2e-5beef491024d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:13:58.219593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f5490c04-926f-47cc-be2e-5beef491024d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:13:58.285398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 502/977] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:13:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:13:58.286288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5490c04-926f-47cc-be2e-5beef491024d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:58.297728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 476/978] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:13:57 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:13:58.303145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53654e8-a47a-47c4-843b-8e33c7e72880 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:58.368719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:58.403676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5490c04-926f-47cc-be2e-5beef491024d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:58.414307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53654e8-a47a-47c4-843b-8e33c7e72880 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:58.432227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:58.441636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce601e-c7ed-401b-8c7d-1a49b11f3e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:58.442271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce601e-c7ed-401b-8c7d-1a49b11f3e44 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.442549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce601e-c7ed-401b-8c7d-1a49b11f3e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.442838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce601e-c7ed-401b-8c7d-1a49b11f3e44 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.443394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce601e-c7ed-401b-8c7d-1a49b11f3e44 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:58.444785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce601e-c7ed-401b-8c7d-1a49b11f3e44 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:58.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a11128-6dd0-4c64-b3a1-b400d204ad07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:58.452542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a11128-6dd0-4c64-b3a1-b400d204ad07 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.452823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a11128-6dd0-4c64-b3a1-b400d204ad07 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.453096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a11128-6dd0-4c64-b3a1-b400d204ad07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-cfce601e-c7ed-401b-8c7d-1a49b11f3e44 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:58.454525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:58.455525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:13:58.458382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 503/979] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:13:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:58.461496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce601e-c7ed-401b-8c7d-1a49b11f3e44 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:58.464865 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a11128-6dd0-4c64-b3a1-b400d204ad07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:58.467790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a11128-6dd0-4c64-b3a1-b400d204ad07 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-00a11128-6dd0-4c64-b3a1-b400d204ad07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:13:58.483822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:58.484799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:13:58.485758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:13:58.485758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:13:58.485758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:58.485758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 477/980] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:13:58.490463 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a11128-6dd0-4c64-b3a1-b400d204ad07 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:58.570825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce601e-c7ed-401b-8c7d-1a49b11f3e44 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:58.596251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a11128-6dd0-4c64-b3a1-b400d204ad07 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:58.604674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78cf14-f093-47bc-aa64-0a7e5a460165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:58.605150 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:58.605357 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78cf14-f093-47bc-aa64-0a7e5a460165 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.605675 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78cf14-f093-47bc-aa64-0a7e5a460165 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.606013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78cf14-f093-47bc-aa64-0a7e5a460165 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.606580 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78cf14-f093-47bc-aa64-0a7e5a460165 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:58.606883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78cf14-f093-47bc-aa64-0a7e5a460165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:58.608882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78cf14-f093-47bc-aa64-0a7e5a460165 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:58.613553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 504/981] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 1 05:13:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:58.614342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78cf14-f093-47bc-aa64-0a7e5a460165 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:58.639113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae547d66-4e24-441f-9c1e-f7a14a37da47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:58.639677 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae547d66-4e24-441f-9c1e-f7a14a37da47 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.639956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae547d66-4e24-441f-9c1e-f7a14a37da47 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.640229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae547d66-4e24-441f-9c1e-f7a14a37da47 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.640871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae547d66-4e24-441f-9c1e-f7a14a37da47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:58.641176 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae547d66-4e24-441f-9c1e-f7a14a37da47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:58.642590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae547d66-4e24-441f-9c1e-f7a14a37da47 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:58.655096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 478/982] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:58.655886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae547d66-4e24-441f-9c1e-f7a14a37da47 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:58.671390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:58.719158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78cf14-f093-47bc-aa64-0a7e5a460165 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:58.754591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b9b69-d6fc-463b-93dd-8c43b0ccc93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:58.755193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b9b69-d6fc-463b-93dd-8c43b0ccc93c None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.755470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b9b69-d6fc-463b-93dd-8c43b0ccc93c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.755741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b9b69-d6fc-463b-93dd-8c43b0ccc93c None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.755840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae547d66-4e24-441f-9c1e-f7a14a37da47 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:58.756385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b9b69-d6fc-463b-93dd-8c43b0ccc93c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:58.757763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b9b69-d6fc-463b-93dd-8c43b0ccc93c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:58.790091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2c3f8-861c-4818-bd1f-e8f8dee35112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:58.790686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2c3f8-861c-4818-bd1f-e8f8dee35112 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.790962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2c3f8-861c-4818-bd1f-e8f8dee35112 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.791228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2c3f8-861c-4818-bd1f-e8f8dee35112 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.791840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2c3f8-861c-4818-bd1f-e8f8dee35112 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:58.792183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2c3f8-861c-4818-bd1f-e8f8dee35112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:58.793653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2c3f8-861c-4818-bd1f-e8f8dee35112 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:58.799195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 479/983] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 1 05:13:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:58.799968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2c3f8-861c-4818-bd1f-e8f8dee35112 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:58.818153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 505/984] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:13:58 2022] POST /identity/v3/services => generated 242 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:13:58.818980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b9b69-d6fc-463b-93dd-8c43b0ccc93c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:58.838972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:58.919850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2c3f8-861c-4818-bd1f-e8f8dee35112 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:58.946958 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b9b69-d6fc-463b-93dd-8c43b0ccc93c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:58.961621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89a800-0959-4a9a-9d57-540eeac22dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:58.962225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:58.962360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89a800-0959-4a9a-9d57-540eeac22dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.962531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89a800-0959-4a9a-9d57-540eeac22dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.962779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89a800-0959-4a9a-9d57-540eeac22dc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.972549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89a800-0959-4a9a-9d57-540eeac22dc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:58.974036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89a800-0959-4a9a-9d57-540eeac22dc0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-4e89a800-0959-4a9a-9d57-540eeac22dc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:13:58.985590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:13:58.986508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:13:58.987866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:13:58.987866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:13:58.987866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:13:58.987866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:13:58.987866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:13:58.987866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:13:58.987866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:13:58.987866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:13:58.988422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642ff18-4d13-4ecb-bf4e-99bfbaa45c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:58.988533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 480/985] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:13:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:13:58.988616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642ff18-4d13-4ecb-bf4e-99bfbaa45c72 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:58.988616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642ff18-4d13-4ecb-bf4e-99bfbaa45c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:58.988616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642ff18-4d13-4ecb-bf4e-99bfbaa45c72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:58.988839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642ff18-4d13-4ecb-bf4e-99bfbaa45c72 None admin] RBAC: Adding query filter params (interface=admin, service_id=a56bda04020c41f6b6f3436a57619f97, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:58.988960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642ff18-4d13-4ecb-bf4e-99bfbaa45c72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:58.990367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642ff18-4d13-4ecb-bf4e-99bfbaa45c72 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:58.992379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89a800-0959-4a9a-9d57-540eeac22dc0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:58.997168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 506/986] 10.222.0.42 () {56 vars in 1249 bytes} [Thu Sep 1 05:13:58 2022] GET /identity/v3/endpoints?service_id=a56bda04020c41f6b6f3436a57619f97&interface=admin®ion_id=RegionOne => generated 193 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:59.003264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642ff18-4d13-4ecb-bf4e-99bfbaa45c72 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:59.095485 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:59.104705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89a800-0959-4a9a-9d57-540eeac22dc0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:59.111450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642ff18-4d13-4ecb-bf4e-99bfbaa45c72 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:59.142662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163cf3b-43fb-46d5-8639-2f2e4f77e89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:59.143271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163cf3b-43fb-46d5-8639-2f2e4f77e89b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:59.143549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163cf3b-43fb-46d5-8639-2f2e4f77e89b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:59.143819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163cf3b-43fb-46d5-8639-2f2e4f77e89b None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:59.144509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163cf3b-43fb-46d5-8639-2f2e4f77e89b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:59.144815 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163cf3b-43fb-46d5-8639-2f2e4f77e89b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:59.146282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163cf3b-43fb-46d5-8639-2f2e4f77e89b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:59.147150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7599b-3b95-4abc-9b3a-23685a68142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:59.147937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7599b-3b95-4abc-9b3a-23685a68142c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:59.148225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7599b-3b95-4abc-9b3a-23685a68142c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:59.148496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7599b-3b95-4abc-9b3a-23685a68142c None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:59.149516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7599b-3b95-4abc-9b3a-23685a68142c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:59.150902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7599b-3b95-4abc-9b3a-23685a68142c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:59.159108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 507/987] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 1 05:13:59 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:13:59.159290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 481/988] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:13:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:13:59.165066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7599b-3b95-4abc-9b3a-23685a68142c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:59.214265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:59.282077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7599b-3b95-4abc-9b3a-23685a68142c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:59.321272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e637d-e5a3-4a8f-92ee-e1929e693453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:59.321909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e637d-e5a3-4a8f-92ee-e1929e693453 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:59.322445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e637d-e5a3-4a8f-92ee-e1929e693453 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:59.322734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e637d-e5a3-4a8f-92ee-e1929e693453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:59.323463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e637d-e5a3-4a8f-92ee-e1929e693453 None admin] RBAC: Adding query filter params (role.id=1830bfc851a04ab3a87c51c2a2c923fb, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=f58aa170e3004197a15dc2dc868d8681) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:59.323771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e637d-e5a3-4a8f-92ee-e1929e693453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:59.325192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e637d-e5a3-4a8f-92ee-e1929e693453 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:59.331712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 508/989] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:59 2022] GET /identity/v3/role_assignments?user.id=f58aa170e3004197a15dc2dc868d8681&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=1830bfc851a04ab3a87c51c2a2c923fb => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:00.807224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd025740-1ba8-4dd6-b59a-cfd40e1603c6 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:00.807743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd025740-1ba8-4dd6-b59a-cfd40e1603c6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:00.807743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd025740-1ba8-4dd6-b59a-cfd40e1603c6 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:00.809252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 482/990] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:00.814377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f571f-643f-43b6-bf20-9e867e2fca7c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:00.814790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f571f-643f-43b6-bf20-9e867e2fca7c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:00.814955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f571f-643f-43b6-bf20-9e867e2fca7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:00.834330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-212f571f-643f-43b6-bf20-9e867e2fca7c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:00.835528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-212f571f-643f-43b6-bf20-9e867e2fca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:00.903939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 509/991] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:01.087885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697db5d-02e3-4d8e-81c9-fca9b27ae81c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.088239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697db5d-02e3-4d8e-81c9-fca9b27ae81c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.088437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697db5d-02e3-4d8e-81c9-fca9b27ae81c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.089944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 483/992] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:01.095265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b02ab-99de-49e1-a743-721f5b642ed5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.095558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b02ab-99de-49e1-a743-721f5b642ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.095834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b02ab-99de-49e1-a743-721f5b642ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.114531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2f3b02ab-99de-49e1-a743-721f5b642ed5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:01.115721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2f3b02ab-99de-49e1-a743-721f5b642ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:01.142710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:01.187673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 510/993] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:01.187870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697db5d-02e3-4d8e-81c9-fca9b27ae81c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:01.279386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1f431-01bd-4a14-ba9a-d5b52bebbc34 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.279744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1f431-01bd-4a14-ba9a-d5b52bebbc34 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.279943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1f431-01bd-4a14-ba9a-d5b52bebbc34 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.281912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 511/994] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:01.286595 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00527c2f-2417-45f6-99da-0169b0453c20 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.286951 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00527c2f-2417-45f6-99da-0169b0453c20 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.287289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00527c2f-2417-45f6-99da-0169b0453c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.298691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697db5d-02e3-4d8e-81c9-fca9b27ae81c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:01.305834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-00527c2f-2417-45f6-99da-0169b0453c20 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:01.307418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-00527c2f-2417-45f6-99da-0169b0453c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:01.335919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabe9cb-64eb-4029-80e8-5b7588b0aaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:01.336545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabe9cb-64eb-4029-80e8-5b7588b0aaef None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.336875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabe9cb-64eb-4029-80e8-5b7588b0aaef None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.337152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabe9cb-64eb-4029-80e8-5b7588b0aaef None admin] PATH_INFO: `/v3/services/compute` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.337785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabe9cb-64eb-4029-80e8-5b7588b0aaef None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:01.339203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabe9cb-64eb-4029-80e8-5b7588b0aaef None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-feabe9cb-64eb-4029-80e8-5b7588b0aaef None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:01.348156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:01.349266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:01.350327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 484/995] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:01.357083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabe9cb-64eb-4029-80e8-5b7588b0aaef None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:01.377162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:01.386524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 512/996] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:01.457661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00527c2f-2417-45f6-99da-0169b0453c20 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:01.460881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabe9cb-64eb-4029-80e8-5b7588b0aaef None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:01.495888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece65796-0cb0-4622-a239-280dc5798987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:01.496488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece65796-0cb0-4622-a239-280dc5798987 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.496764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece65796-0cb0-4622-a239-280dc5798987 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.497036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece65796-0cb0-4622-a239-280dc5798987 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.497690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece65796-0cb0-4622-a239-280dc5798987 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:01.498090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece65796-0cb0-4622-a239-280dc5798987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:01.499462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece65796-0cb0-4622-a239-280dc5798987 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:01.504231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 485/997] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:14:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:01.509736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece65796-0cb0-4622-a239-280dc5798987 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:01.560092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:01.569676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00527c2f-2417-45f6-99da-0169b0453c20 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:01.595309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:01.606513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6db317-149e-4e31-932b-b2bd591299bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:01.607154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6db317-149e-4e31-932b-b2bd591299bd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.607960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6db317-149e-4e31-932b-b2bd591299bd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.608235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6db317-149e-4e31-932b-b2bd591299bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.608913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6db317-149e-4e31-932b-b2bd591299bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:01.610298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6db317-149e-4e31-932b-b2bd591299bd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:01.615194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece65796-0cb0-4622-a239-280dc5798987 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4a6db317-149e-4e31-932b-b2bd591299bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:01.619062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:01.620114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:14:01.620991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:01.625529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 513/998] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 1 05:14:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:14:01.626485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6db317-149e-4e31-932b-b2bd591299bd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:01.651889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59e58e-8d94-41b1-876f-443e6cbf7d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:01.652496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59e58e-8d94-41b1-876f-443e6cbf7d06 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.652770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59e58e-8d94-41b1-876f-443e6cbf7d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.653040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59e58e-8d94-41b1-876f-443e6cbf7d06 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.653697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59e58e-8d94-41b1-876f-443e6cbf7d06 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:01.653998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59e58e-8d94-41b1-876f-443e6cbf7d06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:01.655583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59e58e-8d94-41b1-876f-443e6cbf7d06 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:01.665016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 486/999] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:14:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:01.665016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59e58e-8d94-41b1-876f-443e6cbf7d06 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:01.733280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6db317-149e-4e31-932b-b2bd591299bd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:01.761823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59e58e-8d94-41b1-876f-443e6cbf7d06 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:01.765902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ef69e-c591-44fb-89db-fa1bfea9608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:01.766504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ef69e-c591-44fb-89db-fa1bfea9608e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.766783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ef69e-c591-44fb-89db-fa1bfea9608e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.767048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ef69e-c591-44fb-89db-fa1bfea9608e None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.767661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ef69e-c591-44fb-89db-fa1bfea9608e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:01.767957 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ef69e-c591-44fb-89db-fa1bfea9608e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:01.769306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ef69e-c591-44fb-89db-fa1bfea9608e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:01.774321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 514/1000] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 1 05:14:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:01.775091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ef69e-c591-44fb-89db-fa1bfea9608e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:01.797522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:01.799961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77addbb4-9c4e-4f92-bcb8-f50bbd13ccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:01.800592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77addbb4-9c4e-4f92-bcb8-f50bbd13ccbb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.800869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77addbb4-9c4e-4f92-bcb8-f50bbd13ccbb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.801142 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77addbb4-9c4e-4f92-bcb8-f50bbd13ccbb None admin] PATH_INFO: `/v3/services/image` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.801751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77addbb4-9c4e-4f92-bcb8-f50bbd13ccbb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:01.803123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77addbb4-9c4e-4f92-bcb8-f50bbd13ccbb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-77addbb4-9c4e-4f92-bcb8-f50bbd13ccbb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:01.811877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:01.813471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:01.814435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 487/1001] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Sep 1 05:14:01 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:01.816011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77addbb4-9c4e-4f92-bcb8-f50bbd13ccbb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:01.826483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:01.876526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ef69e-c591-44fb-89db-fa1bfea9608e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:01.910359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed77d4b-bbda-4376-be13-34c148ddbf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:01.910967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed77d4b-bbda-4376-be13-34c148ddbf9f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.911248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed77d4b-bbda-4376-be13-34c148ddbf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.911523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed77d4b-bbda-4376-be13-34c148ddbf9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.912222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed77d4b-bbda-4376-be13-34c148ddbf9f None admin] RBAC: Adding query filter params (interface=public, service_id=015412b3f6ca4ea0892e88b31058e3fc, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:01.912532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed77d4b-bbda-4376-be13-34c148ddbf9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:01.913960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed77d4b-bbda-4376-be13-34c148ddbf9f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:01.918216 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77addbb4-9c4e-4f92-bcb8-f50bbd13ccbb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:01.919025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 515/1002] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 1 05:14:01 2022] GET /identity/v3/endpoints?service_id=015412b3f6ca4ea0892e88b31058e3fc&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:01.920220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed77d4b-bbda-4376-be13-34c148ddbf9f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:01.953271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76f995-7cf1-4224-ac5c-cacf540ae7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:01.953908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76f995-7cf1-4224-ac5c-cacf540ae7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:01.954185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76f995-7cf1-4224-ac5c-cacf540ae7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:01.954457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76f995-7cf1-4224-ac5c-cacf540ae7b5 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:01.955084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76f995-7cf1-4224-ac5c-cacf540ae7b5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:01.955391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76f995-7cf1-4224-ac5c-cacf540ae7b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:01.956755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76f995-7cf1-4224-ac5c-cacf540ae7b5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:01.961537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 488/1003] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 1 05:14:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:01.962338 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76f995-7cf1-4224-ac5c-cacf540ae7b5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:02.006188 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:02.022100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed77d4b-bbda-4376-be13-34c148ddbf9f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:02.036903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:02.054284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9c0e4-5414-4b74-b694-9122f48a2abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:02.054888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9c0e4-5414-4b74-b694-9122f48a2abc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:02.055164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9c0e4-5414-4b74-b694-9122f48a2abc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:02.055433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9c0e4-5414-4b74-b694-9122f48a2abc None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:02.056058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9c0e4-5414-4b74-b694-9122f48a2abc None admin] RBAC: Adding query filter params (name=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:02.056354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9c0e4-5414-4b74-b694-9122f48a2abc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:02.057741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9c0e4-5414-4b74-b694-9122f48a2abc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:02.063312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 516/1004] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:02.064580 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9c0e4-5414-4b74-b694-9122f48a2abc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:02.066678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76f995-7cf1-4224-ac5c-cacf540ae7b5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:02.105909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb7d7f-6b3e-4fa4-8bc5-255b28607960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:02.106503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb7d7f-6b3e-4fa4-8bc5-255b28607960 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:02.106778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb7d7f-6b3e-4fa4-8bc5-255b28607960 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:02.107043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb7d7f-6b3e-4fa4-8bc5-255b28607960 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:02.108412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb7d7f-6b3e-4fa4-8bc5-255b28607960 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:02.109942 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb7d7f-6b3e-4fa4-8bc5-255b28607960 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:02.116315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 489/1005] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 1 05:14:01 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:14:02.117080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb7d7f-6b3e-4fa4-8bc5-255b28607960 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:02.188100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9c0e4-5414-4b74-b694-9122f48a2abc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:02.224796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e78c7-a65f-4fec-808d-9609d1d1b3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:02.225432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e78c7-a65f-4fec-808d-9609d1d1b3ff None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:02.225743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e78c7-a65f-4fec-808d-9609d1d1b3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:02.226015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e78c7-a65f-4fec-808d-9609d1d1b3ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:02.226742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e78c7-a65f-4fec-808d-9609d1d1b3ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:02.228082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e78c7-a65f-4fec-808d-9609d1d1b3ff None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:02.234469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb7d7f-6b3e-4fa4-8bc5-255b28607960 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:02.272685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918e87c-55fa-42bb-8b55-78e753b63cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:02.273324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e87c-55fa-42bb-8b55-78e753b63cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:02.273640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e87c-55fa-42bb-8b55-78e753b63cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:02.273909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e87c-55fa-42bb-8b55-78e753b63cd3 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:02.274542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918e87c-55fa-42bb-8b55-78e753b63cd3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:02.274847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918e87c-55fa-42bb-8b55-78e753b63cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:02.276233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918e87c-55fa-42bb-8b55-78e753b63cd3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:02.278390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:02.281944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 490/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:02.311455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 517/1007] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:14:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:14:04.037909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b100-cbe3-43a3-bf73-a6b7f2eaa06e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:04.038403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b100-cbe3-43a3-bf73-a6b7f2eaa06e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:04.038403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b100-cbe3-43a3-bf73-a6b7f2eaa06e None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:04.039730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 491/1008] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:04.045530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f3e60-b8b3-47e0-9465-98f8b68677e3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:04.045989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f3e60-b8b3-47e0-9465-98f8b68677e3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:04.046222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f3e60-b8b3-47e0-9465-98f8b68677e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:04.064841 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-410f3e60-b8b3-47e0-9465-98f8b68677e3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:04.066114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-410f3e60-b8b3-47e0-9465-98f8b68677e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:04.068337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:04.140161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 518/1009] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:04.252047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a766aee-a6ca-4dcf-88bc-4e9dcd143873 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:04.252362 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a766aee-a6ca-4dcf-88bc-4e9dcd143873 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:04.252600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a766aee-a6ca-4dcf-88bc-4e9dcd143873 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:04.254126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 492/1010] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:14:04.259084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaccf0-7909-43b6-b412-1bde656c8f66 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:04.259373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaccf0-7909-43b6-b412-1bde656c8f66 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:04.259650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaccf0-7909-43b6-b412-1bde656c8f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:04.261682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fdace-3414-4216-aa3c-dce83e3e8a4a None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:04.261973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fdace-3414-4216-aa3c-dce83e3e8a4a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:04.262315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fdace-3414-4216-aa3c-dce83e3e8a4a None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:04.263629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 493/1011] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:14:04.268608 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab3f8e-694d-4971-af12-36ea99b6df8c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:04.268887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab3f8e-694d-4971-af12-36ea99b6df8c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:04.269155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab3f8e-694d-4971-af12-36ea99b6df8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:04.278356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4ecaccf0-7909-43b6-b412-1bde656c8f66 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:04.280080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4ecaccf0-7909-43b6-b412-1bde656c8f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:04.290075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5bab3f8e-694d-4971-af12-36ea99b6df8c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:04.290075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5bab3f8e-694d-4971-af12-36ea99b6df8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:04.355200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 519/1012] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:04.366703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 494/1013] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:04.410961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaccf0-7909-43b6-b412-1bde656c8f66 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:04.475473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:04.514281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaccf0-7909-43b6-b412-1bde656c8f66 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:04.549260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bb31a-92b9-420f-93bc-eb45c117eb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:04.549848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bb31a-92b9-420f-93bc-eb45c117eb6c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:04.550126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bb31a-92b9-420f-93bc-eb45c117eb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:04.550397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bb31a-92b9-420f-93bc-eb45c117eb6c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:04.550974 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bb31a-92b9-420f-93bc-eb45c117eb6c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:04.552331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bb31a-92b9-420f-93bc-eb45c117eb6c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4f0bb31a-92b9-420f-93bc-eb45c117eb6c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:04.561093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:04.565681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:04.566605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 520/1014] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:04.568862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab3f8e-694d-4971-af12-36ea99b6df8c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:04.630450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bb31a-92b9-420f-93bc-eb45c117eb6c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:04.672648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab3f8e-694d-4971-af12-36ea99b6df8c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:04.701155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:04.706831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c476a-2066-4681-bdbd-8d2b646ef4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:04.707439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c476a-2066-4681-bdbd-8d2b646ef4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:04.717240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c476a-2066-4681-bdbd-8d2b646ef4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:04.717240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c476a-2066-4681-bdbd-8d2b646ef4d0 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:04.717240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c476a-2066-4681-bdbd-8d2b646ef4d0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:04.717240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c476a-2066-4681-bdbd-8d2b646ef4d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:04.717240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c476a-2066-4681-bdbd-8d2b646ef4d0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:04.720498 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 495/1015] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:14:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:04.721579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c476a-2066-4681-bdbd-8d2b646ef4d0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:04.735708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bb31a-92b9-420f-93bc-eb45c117eb6c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:04.761654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:04.769583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69195a5-db45-4449-9021-ec8376b3cbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:04.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69195a5-db45-4449-9021-ec8376b3cbb1 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:04.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69195a5-db45-4449-9021-ec8376b3cbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:04.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69195a5-db45-4449-9021-ec8376b3cbb1 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:04.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69195a5-db45-4449-9021-ec8376b3cbb1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:04.775573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69195a5-db45-4449-9021-ec8376b3cbb1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:04.823270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c476a-2066-4681-bdbd-8d2b646ef4d0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:04.856868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176116b-a844-4684-942d-ecefb09e3220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:04.857620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176116b-a844-4684-942d-ecefb09e3220 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:04.857903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176116b-a844-4684-942d-ecefb09e3220 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:04.858177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176116b-a844-4684-942d-ecefb09e3220 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:04.859324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176116b-a844-4684-942d-ecefb09e3220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:04.860756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176116b-a844-4684-942d-ecefb09e3220 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:04.862395 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 496/1016] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:04.863223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176116b-a844-4684-942d-ecefb09e3220 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:04.880841 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 521/1017] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:14:04 2022] POST /identity/v3/services => generated 242 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:14:04.881937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69195a5-db45-4449-9021-ec8376b3cbb1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:04.934432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:04.990254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176116b-a844-4684-942d-ecefb09e3220 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:05.005378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69195a5-db45-4449-9021-ec8376b3cbb1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:05.036814 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45ba87-de96-4200-a337-a7842f38bc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:05.037420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45ba87-de96-4200-a337-a7842f38bc43 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:05.037720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45ba87-de96-4200-a337-a7842f38bc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:05.037991 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45ba87-de96-4200-a337-a7842f38bc43 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:05.038613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45ba87-de96-4200-a337-a7842f38bc43 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:05.038913 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45ba87-de96-4200-a337-a7842f38bc43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:05.040279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45ba87-de96-4200-a337-a7842f38bc43 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:05.045519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 497/1018] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 1 05:14:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:05.047591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece68075-f6fc-4c55-a191-68c594253d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:05.048218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece68075-f6fc-4c55-a191-68c594253d9a None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:05.048489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece68075-f6fc-4c55-a191-68c594253d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:05.048771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece68075-f6fc-4c55-a191-68c594253d9a None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:05.049548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece68075-f6fc-4c55-a191-68c594253d9a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:05.050922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece68075-f6fc-4c55-a191-68c594253d9a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:05.051962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45ba87-de96-4200-a337-a7842f38bc43 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:05.141231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 522/1019] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:14:04 2022] POST /identity/v3/projects => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:14:05.162251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45ba87-de96-4200-a337-a7842f38bc43 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:05.195002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:05.205845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf02498-3305-4869-90be-3d01c1a95056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:05.206523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf02498-3305-4869-90be-3d01c1a95056 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:05.206806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf02498-3305-4869-90be-3d01c1a95056 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:05.207078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf02498-3305-4869-90be-3d01c1a95056 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:05.207761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf02498-3305-4869-90be-3d01c1a95056 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:05.209167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf02498-3305-4869-90be-3d01c1a95056 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:05.299148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 498/1020] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:14:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:14:06.839196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35154d15-a377-4584-8de6-a2b1a0568d4b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:06.839745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35154d15-a377-4584-8de6-a2b1a0568d4b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:06.839745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35154d15-a377-4584-8de6-a2b1a0568d4b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:06.841127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 523/1021] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:06.846407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b21c6-799e-41f5-bb48-587163b7975c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:06.846856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b21c6-799e-41f5-bb48-587163b7975c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:06.847052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b21c6-799e-41f5-bb48-587163b7975c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:06.865863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-341b21c6-799e-41f5-bb48-587163b7975c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:06.867053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-341b21c6-799e-41f5-bb48-587163b7975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:06.944489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 499/1022] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:07.132985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74472470-27cc-42ac-bc3f-40815bb2160a None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:07.133385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74472470-27cc-42ac-bc3f-40815bb2160a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:07.133609 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74472470-27cc-42ac-bc3f-40815bb2160a None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:07.135039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 524/1023] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:07.140223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcf584-9792-49dc-955d-ae9709c1f342 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:07.140513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcf584-9792-49dc-955d-ae9709c1f342 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:07.140789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcf584-9792-49dc-955d-ae9709c1f342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:07.160851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-90bcf584-9792-49dc-955d-ae9709c1f342 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:07.162073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-90bcf584-9792-49dc-955d-ae9709c1f342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:07.218386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74472470-27cc-42ac-bc3f-40815bb2160a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:07.232406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 500/1024] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:14:07.257092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:07.301579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d303e0-de95-47e1-96c6-59fa1c05984f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:07.301866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d303e0-de95-47e1-96c6-59fa1c05984f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:07.302141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d303e0-de95-47e1-96c6-59fa1c05984f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:07.303481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 501/1025] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:14:07.308387 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af2f00-9aac-4be1-a9bb-e9f3b68f18a6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:07.308663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af2f00-9aac-4be1-a9bb-e9f3b68f18a6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:07.308929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af2f00-9aac-4be1-a9bb-e9f3b68f18a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:07.326990 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e2af2f00-9aac-4be1-a9bb-e9f3b68f18a6 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:07.327388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74472470-27cc-42ac-bc3f-40815bb2160a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:07.328160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e2af2f00-9aac-4be1-a9bb-e9f3b68f18a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:07.359048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:07.364751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ecc69-aafd-41e4-a68b-8af443024794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:07.365391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ecc69-aafd-41e4-a68b-8af443024794 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:07.365695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ecc69-aafd-41e4-a68b-8af443024794 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:07.365958 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ecc69-aafd-41e4-a68b-8af443024794 None admin] PATH_INFO: `/v3/services/image` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:07.366536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ecc69-aafd-41e4-a68b-8af443024794 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:07.367959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ecc69-aafd-41e4-a68b-8af443024794 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-db9ecc69-aafd-41e4-a68b-8af443024794 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:07.377127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:07.378406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:07.379445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 525/1026] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Sep 1 05:14:07 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:07.384133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ecc69-aafd-41e4-a68b-8af443024794 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:07.403131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 502/1027] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:14:07.488253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:07.491425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ecc69-aafd-41e4-a68b-8af443024794 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:07.510689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af2f00-9aac-4be1-a9bb-e9f3b68f18a6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:07.526632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86faf00c-8159-437d-8129-eacd580f07eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:07.527250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faf00c-8159-437d-8129-eacd580f07eb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:07.527528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faf00c-8159-437d-8129-eacd580f07eb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:07.527802 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faf00c-8159-437d-8129-eacd580f07eb None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:07.528434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86faf00c-8159-437d-8129-eacd580f07eb None admin] RBAC: Adding query filter params (name=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:07.529313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86faf00c-8159-437d-8129-eacd580f07eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:07.530713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86faf00c-8159-437d-8129-eacd580f07eb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:07.540132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 526/1028] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:07.541754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86faf00c-8159-437d-8129-eacd580f07eb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:07.623085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af2f00-9aac-4be1-a9bb-e9f3b68f18a6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:07.645859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86faf00c-8159-437d-8129-eacd580f07eb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:07.660579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098e312-3726-49bc-9f0d-482deec727c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:07.661211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098e312-3726-49bc-9f0d-482deec727c6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:07.661512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098e312-3726-49bc-9f0d-482deec727c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:07.661795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098e312-3726-49bc-9f0d-482deec727c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:07.662927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098e312-3726-49bc-9f0d-482deec727c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:07.664396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098e312-3726-49bc-9f0d-482deec727c6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:07.666154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 503/1029] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:07.671575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098e312-3726-49bc-9f0d-482deec727c6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:07.685236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329804f-16ec-4815-b397-7c7a7e0145c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:07.685876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329804f-16ec-4815-b397-7c7a7e0145c8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:07.686155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329804f-16ec-4815-b397-7c7a7e0145c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:07.686425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329804f-16ec-4815-b397-7c7a7e0145c8 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:07.687051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329804f-16ec-4815-b397-7c7a7e0145c8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:07.687353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329804f-16ec-4815-b397-7c7a7e0145c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:07.688718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329804f-16ec-4815-b397-7c7a7e0145c8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:07.693417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 527/1030] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:07.694216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329804f-16ec-4815-b397-7c7a7e0145c8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:07.707895 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:07.731329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:07.774753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098e312-3726-49bc-9f0d-482deec727c6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:07.802578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329804f-16ec-4815-b397-7c7a7e0145c8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:07.808680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1940db-5004-427f-932e-e13aede449b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:07.809274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1940db-5004-427f-932e-e13aede449b0 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:07.809588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1940db-5004-427f-932e-e13aede449b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:07.810370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1940db-5004-427f-932e-e13aede449b0 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:07.811322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1940db-5004-427f-932e-e13aede449b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:07.812739 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1940db-5004-427f-932e-e13aede449b0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:07.836779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f2ccd-0900-4121-bad1-e7f62db7b1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:07.837427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f2ccd-0900-4121-bad1-e7f62db7b1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:07.837734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f2ccd-0900-4121-bad1-e7f62db7b1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:07.838005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f2ccd-0900-4121-bad1-e7f62db7b1a6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:07.848674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f2ccd-0900-4121-bad1-e7f62db7b1a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5b5f2ccd-0900-4121-bad1-e7f62db7b1a6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:07.853852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:07.854924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 528/1031] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 1 05:14:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:07.856147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f2ccd-0900-4121-bad1-e7f62db7b1a6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:07.908932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 504/1032] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:14:07 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:14:07.910457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1940db-5004-427f-932e-e13aede449b0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:07.948414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:07.963030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f2ccd-0900-4121-bad1-e7f62db7b1a6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:08.003612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae04e0-2cfa-4cb2-9b48-f08e16cb5df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:08.004330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae04e0-2cfa-4cb2-9b48-f08e16cb5df3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:08.004509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae04e0-2cfa-4cb2-9b48-f08e16cb5df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:08.004775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae04e0-2cfa-4cb2-9b48-f08e16cb5df3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:08.005500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae04e0-2cfa-4cb2-9b48-f08e16cb5df3 None admin] RBAC: Adding query filter params (interface=public, service_id=a55e22d567174da7b2e29b41af518783, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:08.006196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae04e0-2cfa-4cb2-9b48-f08e16cb5df3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:08.007183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae04e0-2cfa-4cb2-9b48-f08e16cb5df3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:08.014762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 529/1033] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 1 05:14:07 2022] GET /identity/v3/endpoints?service_id=a55e22d567174da7b2e29b41af518783&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:08.020390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae04e0-2cfa-4cb2-9b48-f08e16cb5df3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:08.039048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:08.048299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1940db-5004-427f-932e-e13aede449b0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:08.094020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c706f3-e7eb-4656-86fb-450ccd45230b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:08.094572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c706f3-e7eb-4656-86fb-450ccd45230b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:08.094854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c706f3-e7eb-4656-86fb-450ccd45230b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:08.095120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c706f3-e7eb-4656-86fb-450ccd45230b None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:08.097125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c706f3-e7eb-4656-86fb-450ccd45230b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:08.097125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c706f3-e7eb-4656-86fb-450ccd45230b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:08.097534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c706f3-e7eb-4656-86fb-450ccd45230b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:08.107747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 505/1034] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 1 05:14:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:08.114441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c706f3-e7eb-4656-86fb-450ccd45230b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:08.152661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae04e0-2cfa-4cb2-9b48-f08e16cb5df3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:08.188908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77a08d-c813-47b6-b2f2-6e98b8a23afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:08.189493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77a08d-c813-47b6-b2f2-6e98b8a23afd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:08.193187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77a08d-c813-47b6-b2f2-6e98b8a23afd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:08.193187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77a08d-c813-47b6-b2f2-6e98b8a23afd None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:08.193187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f77a08d-c813-47b6-b2f2-6e98b8a23afd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:08.193187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f77a08d-c813-47b6-b2f2-6e98b8a23afd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:08.199320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 530/1035] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 1 05:14:08 2022] GET /identity/v3/services => generated 1957 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:14:08.226255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c706f3-e7eb-4656-86fb-450ccd45230b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:08.266541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266668e-19b5-4cc5-ad2c-490ecdefc5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:08.267194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266668e-19b5-4cc5-ad2c-490ecdefc5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:08.267471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266668e-19b5-4cc5-ad2c-490ecdefc5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:08.267740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266668e-19b5-4cc5-ad2c-490ecdefc5c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:08.280500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266668e-19b5-4cc5-ad2c-490ecdefc5c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:08.282016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266668e-19b5-4cc5-ad2c-490ecdefc5c3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e266668e-19b5-4cc5-ad2c-490ecdefc5c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:08.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:08.298548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:08.299866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:08.299866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:08.299866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:08.299866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 506/1036] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:14:08.304559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77a08d-c813-47b6-b2f2-6e98b8a23afd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:08.329067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:08.416099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77a08d-c813-47b6-b2f2-6e98b8a23afd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:08.451397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6d9cf-64c6-47f6-89d7-2b302c13b340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:08.451923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6d9cf-64c6-47f6-89d7-2b302c13b340 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:08.452197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6d9cf-64c6-47f6-89d7-2b302c13b340 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:08.452464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6d9cf-64c6-47f6-89d7-2b302c13b340 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:08.453095 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6d9cf-64c6-47f6-89d7-2b302c13b340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:08.453391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6d9cf-64c6-47f6-89d7-2b302c13b340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:08.454756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6d9cf-64c6-47f6-89d7-2b302c13b340 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:08.467527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 531/1037] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:08.473074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266668e-19b5-4cc5-ad2c-490ecdefc5c3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:08.482797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:08.574875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266668e-19b5-4cc5-ad2c-490ecdefc5c3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:08.608365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff1377-d5db-41eb-b030-cc7de7ec861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:08.609179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff1377-d5db-41eb-b030-cc7de7ec861a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:08.609544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff1377-d5db-41eb-b030-cc7de7ec861a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:08.609818 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff1377-d5db-41eb-b030-cc7de7ec861a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:08.617427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff1377-d5db-41eb-b030-cc7de7ec861a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:08.618882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff1377-d5db-41eb-b030-cc7de7ec861a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e1ff1377-d5db-41eb-b030-cc7de7ec861a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:14:08.630850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:08.631847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:08.632844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:08.632844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:08.632844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:14:08.632844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:14:08.632844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:14:08.632844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:14:08.632844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:14:08.632844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:14:08.632844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:14:08.632844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:08.636641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 507/1038] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 1 05:14:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:08.638633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6d9cf-64c6-47f6-89d7-2b302c13b340 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:08.725760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:08.749965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6d9cf-64c6-47f6-89d7-2b302c13b340 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:08.786067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263229f4-9b76-4f91-92e1-4f613c7b09af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:08.786676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263229f4-9b76-4f91-92e1-4f613c7b09af None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:08.786954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263229f4-9b76-4f91-92e1-4f613c7b09af None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:08.787227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263229f4-9b76-4f91-92e1-4f613c7b09af None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:08.787949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263229f4-9b76-4f91-92e1-4f613c7b09af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:08.788253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263229f4-9b76-4f91-92e1-4f613c7b09af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:08.789717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263229f4-9b76-4f91-92e1-4f613c7b09af None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:08.812087 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 532/1039] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 1 05:14:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:08.813804 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff1377-d5db-41eb-b030-cc7de7ec861a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:08.889347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:08.935868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff1377-d5db-41eb-b030-cc7de7ec861a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:08.971858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bddb3-05ee-41cb-85cc-df82b8d6a3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:08.972458 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bddb3-05ee-41cb-85cc-df82b8d6a3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:08.972732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bddb3-05ee-41cb-85cc-df82b8d6a3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:08.972997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bddb3-05ee-41cb-85cc-df82b8d6a3e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:09.040307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bddb3-05ee-41cb-85cc-df82b8d6a3e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:09.041692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bddb3-05ee-41cb-85cc-df82b8d6a3e8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:09.108788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:09.148379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 508/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:08 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:14:09.149558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263229f4-9b76-4f91-92e1-4f613c7b09af None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:09.264762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263229f4-9b76-4f91-92e1-4f613c7b09af None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:09.300353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bd542-866b-4b07-80d9-6d7a8c452d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:09.301257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bd542-866b-4b07-80d9-6d7a8c452d4c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:09.301728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bd542-866b-4b07-80d9-6d7a8c452d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:09.302184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bd542-866b-4b07-80d9-6d7a8c452d4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:09.304621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:09.314600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502bd542-866b-4b07-80d9-6d7a8c452d4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:09.316284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502bd542-866b-4b07-80d9-6d7a8c452d4c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-502bd542-866b-4b07-80d9-6d7a8c452d4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:09.331649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:09.332701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:09.338017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:09.338017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:09.338017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:09.338017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 533/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:09.340074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bddb3-05ee-41cb-85cc-df82b8d6a3e8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:09.444356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bddb3-05ee-41cb-85cc-df82b8d6a3e8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:09.480426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122004e-cbdc-4453-afcd-89b81611451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:09.481346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122004e-cbdc-4453-afcd-89b81611451d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:09.482104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122004e-cbdc-4453-afcd-89b81611451d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:09.482571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122004e-cbdc-4453-afcd-89b81611451d None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:09.483391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122004e-cbdc-4453-afcd-89b81611451d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:09.483879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122004e-cbdc-4453-afcd-89b81611451d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:09.485450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122004e-cbdc-4453-afcd-89b81611451d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:09.497848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 509/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:09.503844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bd542-866b-4b07-80d9-6d7a8c452d4c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:09.615991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bd542-866b-4b07-80d9-6d7a8c452d4c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:09.649781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74fdd7-8819-4f12-8c7c-2e0ac9ec8244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:09.650415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74fdd7-8819-4f12-8c7c-2e0ac9ec8244 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:09.650693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74fdd7-8819-4f12-8c7c-2e0ac9ec8244 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:09.650964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74fdd7-8819-4f12-8c7c-2e0ac9ec8244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:09.657775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74fdd7-8819-4f12-8c7c-2e0ac9ec8244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:09.659214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74fdd7-8819-4f12-8c7c-2e0ac9ec8244 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0f74fdd7-8819-4f12-8c7c-2e0ac9ec8244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:14:09.671141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:09.672065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:09.677717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:14:09.677717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:14:09.677717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:14:09.677717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:14:09.677717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:14:09.677717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:14:09.677717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:09.677717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:09.677717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 534/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:14:09.679865 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122004e-cbdc-4453-afcd-89b81611451d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:09.711000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:09.796190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122004e-cbdc-4453-afcd-89b81611451d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:09.830985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782e683-5cc9-47d6-a04c-9e5dcceaad07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:09.831588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782e683-5cc9-47d6-a04c-9e5dcceaad07 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:09.831861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782e683-5cc9-47d6-a04c-9e5dcceaad07 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:09.832127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782e683-5cc9-47d6-a04c-9e5dcceaad07 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:09.832808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782e683-5cc9-47d6-a04c-9e5dcceaad07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:09.833109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782e683-5cc9-47d6-a04c-9e5dcceaad07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:09.834551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782e683-5cc9-47d6-a04c-9e5dcceaad07 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:09.852288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 510/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:14:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:09.853699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74fdd7-8819-4f12-8c7c-2e0ac9ec8244 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:09.908349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:09.921772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c79e2-ba14-497e-8d9e-afb1d366920a None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:09.922087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c79e2-ba14-497e-8d9e-afb1d366920a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:09.922334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c79e2-ba14-497e-8d9e-afb1d366920a None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:09.923685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 511/1045] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:14:09.928869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834947f7-abb2-4e12-8c13-6dbbc4eba8b3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:09.929152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834947f7-abb2-4e12-8c13-6dbbc4eba8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:09.929434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834947f7-abb2-4e12-8c13-6dbbc4eba8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:09.950413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-834947f7-abb2-4e12-8c13-6dbbc4eba8b3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:09.951749 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-834947f7-abb2-4e12-8c13-6dbbc4eba8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:09.965403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74fdd7-8819-4f12-8c7c-2e0ac9ec8244 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:10.003674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232c3ec-6b1c-41eb-9e1c-7d99df45f50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:10.004457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232c3ec-6b1c-41eb-9e1c-7d99df45f50d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:10.004735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232c3ec-6b1c-41eb-9e1c-7d99df45f50d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:10.005007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232c3ec-6b1c-41eb-9e1c-7d99df45f50d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:10.005794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232c3ec-6b1c-41eb-9e1c-7d99df45f50d None admin] RBAC: Adding query filter params (role.id=1830bfc851a04ab3a87c51c2a2c923fb, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=7cda3c1f67e54ee08abc014399b07cc5) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:10.006100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232c3ec-6b1c-41eb-9e1c-7d99df45f50d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:10.007532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232c3ec-6b1c-41eb-9e1c-7d99df45f50d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:10.014589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 535/1046] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:09 2022] GET /identity/v3/role_assignments?user.id=7cda3c1f67e54ee08abc014399b07cc5&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=1830bfc851a04ab3a87c51c2a2c923fb => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:10.036774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 512/1047] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:14:10.149878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aaffd-9e60-49f1-af61-f36704490d1f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:10.150387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aaffd-9e60-49f1-af61-f36704490d1f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:10.150387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aaffd-9e60-49f1-af61-f36704490d1f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:10.151872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 536/1048] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:10.156901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a1464-b12b-422a-b135-58be069e6253 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:10.157188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a1464-b12b-422a-b135-58be069e6253 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:10.157487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a1464-b12b-422a-b135-58be069e6253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:10.176480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-414a1464-b12b-422a-b135-58be069e6253 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:10.177735 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-414a1464-b12b-422a-b135-58be069e6253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:10.190619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:10.256012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 513/1049] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:10.315845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4aaffd-9e60-49f1-af61-f36704490d1f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:10.422017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4aaffd-9e60-49f1-af61-f36704490d1f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:10.432578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:10.457727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfa73b-2ddc-45fe-8c78-341a88b7e86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:10.458597 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfa73b-2ddc-45fe-8c78-341a88b7e86a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:10.459025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfa73b-2ddc-45fe-8c78-341a88b7e86a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:10.459447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfa73b-2ddc-45fe-8c78-341a88b7e86a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:10.471687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfa73b-2ddc-45fe-8c78-341a88b7e86a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-cccfa73b-2ddc-45fe-8c78-341a88b7e86a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:10.476631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:10.477791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:10.478879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:10.480899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 537/1050] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:14:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:10.487240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a1464-b12b-422a-b135-58be069e6253 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:10.520441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfa73b-2ddc-45fe-8c78-341a88b7e86a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:10.593441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a1464-b12b-422a-b135-58be069e6253 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:10.623549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfa73b-2ddc-45fe-8c78-341a88b7e86a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:10.628366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40baa54d-f8ff-48df-9101-feffe86f9e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:10.628966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40baa54d-f8ff-48df-9101-feffe86f9e0c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:10.629244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40baa54d-f8ff-48df-9101-feffe86f9e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:10.629549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40baa54d-f8ff-48df-9101-feffe86f9e0c None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:10.630208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40baa54d-f8ff-48df-9101-feffe86f9e0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:10.630504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40baa54d-f8ff-48df-9101-feffe86f9e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:10.631867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40baa54d-f8ff-48df-9101-feffe86f9e0c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:10.640928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 514/1051] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:14:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:10.646633 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40baa54d-f8ff-48df-9101-feffe86f9e0c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:10.658061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:10.660390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b8662-9ed1-4806-8f11-77a22a619909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:10.661080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b8662-9ed1-4806-8f11-77a22a619909 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:10.661295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b8662-9ed1-4806-8f11-77a22a619909 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:10.661596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b8662-9ed1-4806-8f11-77a22a619909 None admin] PATH_INFO: `/v3/services/image` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:10.662169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b8662-9ed1-4806-8f11-77a22a619909 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:10.663520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b8662-9ed1-4806-8f11-77a22a619909 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5f2b8662-9ed1-4806-8f11-77a22a619909 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:10.673437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:10.674478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:14:10.675361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:10.676941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 538/1052] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Sep 1 05:14:10 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:14:10.680494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b8662-9ed1-4806-8f11-77a22a619909 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:10.720811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:10.763091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40baa54d-f8ff-48df-9101-feffe86f9e0c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:10.791025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b8662-9ed1-4806-8f11-77a22a619909 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:10.800965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b5b97-ee4e-4c6a-b3a1-0a47d0086943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:10.801958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b5b97-ee4e-4c6a-b3a1-0a47d0086943 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:10.802242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b5b97-ee4e-4c6a-b3a1-0a47d0086943 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:10.802508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b5b97-ee4e-4c6a-b3a1-0a47d0086943 None admin] PATH_INFO: `/v3/users/fdf9ce5d064148e78b75230e46a01759` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:10.817789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b5b97-ee4e-4c6a-b3a1-0a47d0086943 None admin] RBAC: Authorizing `identity:get_user(user_id=fdf9ce5d064148e78b75230e46a01759)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:10.819384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b5b97-ee4e-4c6a-b3a1-0a47d0086943 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:10.821175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 515/1053] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:10 2022] GET /identity/v3/users/fdf9ce5d064148e78b75230e46a01759 => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:10.827699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b5b97-ee4e-4c6a-b3a1-0a47d0086943 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:10.829497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4a3bb-733d-4b08-9a7b-d1a7ac8fa79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:10.830115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4a3bb-733d-4b08-9a7b-d1a7ac8fa79c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:10.830395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4a3bb-733d-4b08-9a7b-d1a7ac8fa79c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:10.830668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4a3bb-733d-4b08-9a7b-d1a7ac8fa79c None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:10.831295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4a3bb-733d-4b08-9a7b-d1a7ac8fa79c None admin] RBAC: Adding query filter params (name=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:10.831595 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4a3bb-733d-4b08-9a7b-d1a7ac8fa79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:10.832946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4a3bb-733d-4b08-9a7b-d1a7ac8fa79c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:10.843355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 539/1054] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:10.844466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4a3bb-733d-4b08-9a7b-d1a7ac8fa79c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:10.904455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:10.940432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b5b97-ee4e-4c6a-b3a1-0a47d0086943 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:10.954588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:10.958838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4a3bb-733d-4b08-9a7b-d1a7ac8fa79c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:10.979274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bcf3f-0b94-4c15-a184-4e27babd14b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:10.979905 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bcf3f-0b94-4c15-a184-4e27babd14b0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:10.980301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bcf3f-0b94-4c15-a184-4e27babd14b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:10.980579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bcf3f-0b94-4c15-a184-4e27babd14b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:10.998304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09483635-4897-4b61-b8df-dd5ea5f3ba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:10.998926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09483635-4897-4b61-b8df-dd5ea5f3ba42 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:10.999208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09483635-4897-4b61-b8df-dd5ea5f3ba42 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:10.999482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09483635-4897-4b61-b8df-dd5ea5f3ba42 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:11.000120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09483635-4897-4b61-b8df-dd5ea5f3ba42 None admin] RBAC: Adding query filter params (type=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:11.000426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09483635-4897-4b61-b8df-dd5ea5f3ba42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:11.001824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09483635-4897-4b61-b8df-dd5ea5f3ba42 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:11.006983 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 540/1055] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:11.012723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09483635-4897-4b61-b8df-dd5ea5f3ba42 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:11.061076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bcf3f-0b94-4c15-a184-4e27babd14b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:11.062598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bcf3f-0b94-4c15-a184-4e27babd14b0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:11.125045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09483635-4897-4b61-b8df-dd5ea5f3ba42 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:11.134780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:11.167428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d641bb-a9c2-4373-9e1b-d86cf4205b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:11.168053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d641bb-a9c2-4373-9e1b-d86cf4205b8a None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:11.168633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d641bb-a9c2-4373-9e1b-d86cf4205b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:11.168904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d641bb-a9c2-4373-9e1b-d86cf4205b8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:11.170507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d641bb-a9c2-4373-9e1b-d86cf4205b8a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:11.171908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d641bb-a9c2-4373-9e1b-d86cf4205b8a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:11.177733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 516/1056] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 05:14:11.183309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bcf3f-0b94-4c15-a184-4e27babd14b0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:11.286279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bcf3f-0b94-4c15-a184-4e27babd14b0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:11.299292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 541/1057] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:14:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:14:11.344316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7ea38-e364-475c-a6b6-a9b8d1fc35c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:11.345016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7ea38-e364-475c-a6b6-a9b8d1fc35c2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:11.345292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7ea38-e364-475c-a6b6-a9b8d1fc35c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:11.345591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7ea38-e364-475c-a6b6-a9b8d1fc35c2 None admin] PATH_INFO: `/v3/projects/3b77605fc7624a358a0cfed28e4c55a7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:11.346832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7ea38-e364-475c-a6b6-a9b8d1fc35c2 None admin] RBAC: Authorizing `identity:get_project(project_id=3b77605fc7624a358a0cfed28e4c55a7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:11.348381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7ea38-e364-475c-a6b6-a9b8d1fc35c2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:11.350415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 517/1058] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:14:11 2022] GET /identity/v3/projects/3b77605fc7624a358a0cfed28e4c55a7 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:11.356941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d641bb-a9c2-4373-9e1b-d86cf4205b8a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:11.378148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:11.472826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d641bb-a9c2-4373-9e1b-d86cf4205b8a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:11.508996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119795c4-128f-4f6d-9e46-aba1d15ee02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:11.509643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119795c4-128f-4f6d-9e46-aba1d15ee02a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:11.509920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119795c4-128f-4f6d-9e46-aba1d15ee02a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:11.510189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119795c4-128f-4f6d-9e46-aba1d15ee02a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:11.510922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119795c4-128f-4f6d-9e46-aba1d15ee02a None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=3b77605fc7624a358a0cfed28e4c55a7, user.id=fdf9ce5d064148e78b75230e46a01759) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:11.511250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119795c4-128f-4f6d-9e46-aba1d15ee02a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:11.512666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119795c4-128f-4f6d-9e46-aba1d15ee02a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:11.519427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 542/1059] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:11 2022] GET /identity/v3/role_assignments?user.id=fdf9ce5d064148e78b75230e46a01759&scope.project.id=3b77605fc7624a358a0cfed28e4c55a7&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:11.936852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cd583-86dc-4ead-8d36-c0df37412543 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:11.937211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cd583-86dc-4ead-8d36-c0df37412543 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:11.937402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cd583-86dc-4ead-8d36-c0df37412543 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:11.938939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 518/1060] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:11.943866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a882deb-2ae6-4426-8be3-e4127ce2d27b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:11.944152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a882deb-2ae6-4426-8be3-e4127ce2d27b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:11.944428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a882deb-2ae6-4426-8be3-e4127ce2d27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:11.963514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1a882deb-2ae6-4426-8be3-e4127ce2d27b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:11.964710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1a882deb-2ae6-4426-8be3-e4127ce2d27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:12.035382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 543/1061] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:12.306354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cd583-86dc-4ead-8d36-c0df37412543 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:12.338407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:12.415322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cd583-86dc-4ead-8d36-c0df37412543 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:12.452741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a666fa-440f-48a4-a20b-d6f722a9a658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:12.453396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a666fa-440f-48a4-a20b-d6f722a9a658 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:12.453705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a666fa-440f-48a4-a20b-d6f722a9a658 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:12.453976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a666fa-440f-48a4-a20b-d6f722a9a658 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:12.465448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a666fa-440f-48a4-a20b-d6f722a9a658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-f8a666fa-440f-48a4-a20b-d6f722a9a658 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:12.471134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:12.472091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:12.472963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 519/1062] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 1 05:14:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:12.479354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a882deb-2ae6-4426-8be3-e4127ce2d27b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:12.494148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:12.582450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a882deb-2ae6-4426-8be3-e4127ce2d27b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:12.618818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2d48c-24d2-4a7e-86fa-89bd195f97d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:12.619426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2d48c-24d2-4a7e-86fa-89bd195f97d0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:12.619703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2d48c-24d2-4a7e-86fa-89bd195f97d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:12.619976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2d48c-24d2-4a7e-86fa-89bd195f97d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:12.620633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2d48c-24d2-4a7e-86fa-89bd195f97d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:12.620935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2d48c-24d2-4a7e-86fa-89bd195f97d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:12.622355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2d48c-24d2-4a7e-86fa-89bd195f97d0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:12.634841 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 544/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 1 05:14:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:12.635960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a666fa-440f-48a4-a20b-d6f722a9a658 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:12.734792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:12.744457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a666fa-440f-48a4-a20b-d6f722a9a658 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:12.781372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd17018-0999-408f-83af-925c8e8cefff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:12.782247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd17018-0999-408f-83af-925c8e8cefff None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:12.782707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd17018-0999-408f-83af-925c8e8cefff None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:12.783122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd17018-0999-408f-83af-925c8e8cefff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:12.793282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd17018-0999-408f-83af-925c8e8cefff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:12.795127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd17018-0999-408f-83af-925c8e8cefff None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-6fd17018-0999-408f-83af-925c8e8cefff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:12.811068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:12.812148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:12.813064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:12.813064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:12.813064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:12.813252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 520/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:14:12.818485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2d48c-24d2-4a7e-86fa-89bd195f97d0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:12.905015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:12.930940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2d48c-24d2-4a7e-86fa-89bd195f97d0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:12.965549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd28a9-bcc0-45f7-bff1-2bc7d5fd1df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:12.966829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd28a9-bcc0-45f7-bff1-2bc7d5fd1df4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:12.967109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd28a9-bcc0-45f7-bff1-2bc7d5fd1df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:12.967380 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd28a9-bcc0-45f7-bff1-2bc7d5fd1df4 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:12.968018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd28a9-bcc0-45f7-bff1-2bc7d5fd1df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:12.968319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd28a9-bcc0-45f7-bff1-2bc7d5fd1df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:12.969699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd28a9-bcc0-45f7-bff1-2bc7d5fd1df4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:12.981866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 545/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:12.987279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd17018-0999-408f-83af-925c8e8cefff None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:13.090093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd17018-0999-408f-83af-925c8e8cefff None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:13.127650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22bc88-6f9e-46dd-bbe7-6ef650af8c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:13.128287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22bc88-6f9e-46dd-bbe7-6ef650af8c40 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.128566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22bc88-6f9e-46dd-bbe7-6ef650af8c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.128839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22bc88-6f9e-46dd-bbe7-6ef650af8c40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.136980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22bc88-6f9e-46dd-bbe7-6ef650af8c40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:13.139102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22bc88-6f9e-46dd-bbe7-6ef650af8c40 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-af22bc88-6f9e-46dd-bbe7-6ef650af8c40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:14:13.150220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:13.151203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:13.152172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:13.152172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:13.152172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:14:13.152172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:14:13.152172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:14:13.152172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:14:13.152172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:14:13.152172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:14:13.152172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:14:13.152172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:13.153356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 521/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 1 05:14:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:13.157497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd28a9-bcc0-45f7-bff1-2bc7d5fd1df4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:13.275911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd28a9-bcc0-45f7-bff1-2bc7d5fd1df4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:13.310585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61adde7-7877-43b2-8481-a9b7c9783f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:13.311494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61adde7-7877-43b2-8481-a9b7c9783f4e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.311778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61adde7-7877-43b2-8481-a9b7c9783f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.312049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61adde7-7877-43b2-8481-a9b7c9783f4e None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.312775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61adde7-7877-43b2-8481-a9b7c9783f4e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:13.313079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61adde7-7877-43b2-8481-a9b7c9783f4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:13.314518 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61adde7-7877-43b2-8481-a9b7c9783f4e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:13.316201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:13.319381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd449-155d-4449-ada7-db449ddee6c0 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.319836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd449-155d-4449-ada7-db449ddee6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.319936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd449-155d-4449-ada7-db449ddee6c0 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.321373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 522/1067] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:14:13.327263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5625-d358-4340-b182-109dd9a6c294 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.327540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5625-d358-4340-b182-109dd9a6c294 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.327810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5625-d358-4340-b182-109dd9a6c294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.334123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:13.338921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 546/1068] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 1 05:14:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:13.339666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61adde7-7877-43b2-8481-a9b7c9783f4e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:13.348037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2b0e5625-d358-4340-b182-109dd9a6c294 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:13.349655 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2b0e5625-d358-4340-b182-109dd9a6c294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:13.421356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 523/1069] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:14:13.432662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcedddd-8456-4608-82e9-01020bb61548 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.432944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcedddd-8456-4608-82e9-01020bb61548 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.433215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcedddd-8456-4608-82e9-01020bb61548 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.435057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 524/1070] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:14:13.440560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76829ce-252f-4b1c-b8a3-36fa279015d3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.440855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76829ce-252f-4b1c-b8a3-36fa279015d3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.441128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76829ce-252f-4b1c-b8a3-36fa279015d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.447232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61adde7-7877-43b2-8481-a9b7c9783f4e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:13.461894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e76829ce-252f-4b1c-b8a3-36fa279015d3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:13.463390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e76829ce-252f-4b1c-b8a3-36fa279015d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:13.490399 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd2e2a-37ba-4457-bf27-0a0d9e48cc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:13.491057 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd2e2a-37ba-4457-bf27-0a0d9e48cc9c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.491337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd2e2a-37ba-4457-bf27-0a0d9e48cc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.491607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd2e2a-37ba-4457-bf27-0a0d9e48cc9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.503221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd2e2a-37ba-4457-bf27-0a0d9e48cc9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:13.504677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd2e2a-37ba-4457-bf27-0a0d9e48cc9c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a5cd2e2a-37ba-4457-bf27-0a0d9e48cc9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:13.521817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:13.522863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:13.523795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:13.523795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:13.523795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:13.523795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 547/1071] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:13.529438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd2e2a-37ba-4457-bf27-0a0d9e48cc9c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:13.539887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 525/1072] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:14:13.579962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:13.634513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd2e2a-37ba-4457-bf27-0a0d9e48cc9c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:13.667665 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5117687-3ac9-44b2-bea5-9bfe252f87e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:13.668271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5117687-3ac9-44b2-bea5-9bfe252f87e5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.668550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5117687-3ac9-44b2-bea5-9bfe252f87e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.669347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5117687-3ac9-44b2-bea5-9bfe252f87e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.670018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5117687-3ac9-44b2-bea5-9bfe252f87e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:13.670321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5117687-3ac9-44b2-bea5-9bfe252f87e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:13.671723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5117687-3ac9-44b2-bea5-9bfe252f87e5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:13.683352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 548/1073] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:13.688882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76829ce-252f-4b1c-b8a3-36fa279015d3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:13.718669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5117687-3ac9-44b2-bea5-9bfe252f87e5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:13.753464 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:13.791955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76829ce-252f-4b1c-b8a3-36fa279015d3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:13.832165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db06ab-e300-4178-9e6b-94f1ccc8c2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:13.832788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db06ab-e300-4178-9e6b-94f1ccc8c2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.832890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5117687-3ac9-44b2-bea5-9bfe252f87e5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:13.833065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db06ab-e300-4178-9e6b-94f1ccc8c2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.833332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db06ab-e300-4178-9e6b-94f1ccc8c2f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.835165 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:13.840271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db06ab-e300-4178-9e6b-94f1ccc8c2f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:13.842160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db06ab-e300-4178-9e6b-94f1ccc8c2f7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d0db06ab-e300-4178-9e6b-94f1ccc8c2f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:14:13.853132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:13.854112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:13.855010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:14:13.855010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:14:13.855010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:14:13.855010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:14:13.855010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:14:13.855010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:14:13.855010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:13.855010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:13.855010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 526/1074] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:13.855836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db06ab-e300-4178-9e6b-94f1ccc8c2f7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:13.874933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0d6c4-7fc9-4a29-a59a-d0f28f92780a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:13.875544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0d6c4-7fc9-4a29-a59a-d0f28f92780a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.875817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0d6c4-7fc9-4a29-a59a-d0f28f92780a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.876085 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0d6c4-7fc9-4a29-a59a-d0f28f92780a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.943697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0d6c4-7fc9-4a29-a59a-d0f28f92780a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:13.945039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0d6c4-7fc9-4a29-a59a-d0f28f92780a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:13.961067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db06ab-e300-4178-9e6b-94f1ccc8c2f7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:13.983679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:13.995114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3305701-7410-42a3-b293-c6d5d6e455c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:13.995732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3305701-7410-42a3-b293-c6d5d6e455c3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.996013 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3305701-7410-42a3-b293-c6d5d6e455c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.996274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3305701-7410-42a3-b293-c6d5d6e455c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:14.007172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3305701-7410-42a3-b293-c6d5d6e455c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-b3305701-7410-42a3-b293-c6d5d6e455c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:14.012381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:14.014094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:14.015055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:14.017103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 527/1075] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:14.018518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3305701-7410-42a3-b293-c6d5d6e455c3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:14.048370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 549/1076] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 05:14:14.049383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0d6c4-7fc9-4a29-a59a-d0f28f92780a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:14.052324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:14.133968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3305701-7410-42a3-b293-c6d5d6e455c3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:14.154079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0d6c4-7fc9-4a29-a59a-d0f28f92780a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:14.168853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bd88b-267c-42c9-91ac-3dfdbae89213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:14.169721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bd88b-267c-42c9-91ac-3dfdbae89213 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:14.169997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bd88b-267c-42c9-91ac-3dfdbae89213 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:14.170267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bd88b-267c-42c9-91ac-3dfdbae89213 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:14.170947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bd88b-267c-42c9-91ac-3dfdbae89213 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:14.171248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bd88b-267c-42c9-91ac-3dfdbae89213 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:14.172660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bd88b-267c-42c9-91ac-3dfdbae89213 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:14.184186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 528/1077] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:14:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:14.185359 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bd88b-267c-42c9-91ac-3dfdbae89213 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:14.186756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d300d-2a1a-4211-983a-56a13f10e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:14.187368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d300d-2a1a-4211-983a-56a13f10e121 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:14.187645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d300d-2a1a-4211-983a-56a13f10e121 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:14.187912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d300d-2a1a-4211-983a-56a13f10e121 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:14.188569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d300d-2a1a-4211-983a-56a13f10e121 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:14.188930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d300d-2a1a-4211-983a-56a13f10e121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:14.190306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d300d-2a1a-4211-983a-56a13f10e121 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:14.209522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 550/1078] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:14:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:14.209522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d300d-2a1a-4211-983a-56a13f10e121 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:14.237612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:14.265584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:14.295563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bd88b-267c-42c9-91ac-3dfdbae89213 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:14.299606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d300d-2a1a-4211-983a-56a13f10e121 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:14.330982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8463997-08d6-4fa1-b94e-2d775cb2735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:14.331606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8463997-08d6-4fa1-b94e-2d775cb2735b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:14.331945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8463997-08d6-4fa1-b94e-2d775cb2735b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:14.332156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8463997-08d6-4fa1-b94e-2d775cb2735b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:14.334820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699a764-e03d-4c9f-81ba-6867919a2dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:14.335532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699a764-e03d-4c9f-81ba-6867919a2dd1 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:14.335848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699a764-e03d-4c9f-81ba-6867919a2dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:14.336125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699a764-e03d-4c9f-81ba-6867919a2dd1 None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/7cda3c1f67e54ee08abc014399b07cc5/roles/1830bfc851a04ab3a87c51c2a2c923fb` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:14.338300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699a764-e03d-4c9f-81ba-6867919a2dd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=7cda3c1f67e54ee08abc014399b07cc5, role_id=1830bfc851a04ab3a87c51c2a2c923fb)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:14.339729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699a764-e03d-4c9f-81ba-6867919a2dd1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:14.343065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8463997-08d6-4fa1-b94e-2d775cb2735b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:14.344503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8463997-08d6-4fa1-b94e-2d775cb2735b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-b8463997-08d6-4fa1-b94e-2d775cb2735b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:14.358844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:14.359992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:14.361148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:14.361148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:14.361148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:14.361148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 529/1079] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:14.361148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8463997-08d6-4fa1-b94e-2d775cb2735b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:14.400266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 551/1080] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:14:14 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/7cda3c1f67e54ee08abc014399b07cc5/roles/1830bfc851a04ab3a87c51c2a2c923fb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:14:14.401086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699a764-e03d-4c9f-81ba-6867919a2dd1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:14.463421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:14.473071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8463997-08d6-4fa1-b94e-2d775cb2735b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:14.514784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560b8f-9667-4112-88c6-7b3c708f1356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:14.515588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560b8f-9667-4112-88c6-7b3c708f1356 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:14.515866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560b8f-9667-4112-88c6-7b3c708f1356 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:14.516133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560b8f-9667-4112-88c6-7b3c708f1356 None admin] PATH_INFO: `/v3/users/fdf9ce5d064148e78b75230e46a01759` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:14.517545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80560b8f-9667-4112-88c6-7b3c708f1356 None admin] RBAC: Authorizing `identity:get_user(user_id=fdf9ce5d064148e78b75230e46a01759)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:14.519462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80560b8f-9667-4112-88c6-7b3c708f1356 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:14.526107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 530/1081] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:14 2022] GET /identity/v3/users/fdf9ce5d064148e78b75230e46a01759 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:14.527096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560b8f-9667-4112-88c6-7b3c708f1356 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:14.538738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699a764-e03d-4c9f-81ba-6867919a2dd1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:14.571300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:14.581421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e1db2-c5a7-4c5e-8541-802a0cc37240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:14.582977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e1db2-c5a7-4c5e-8541-802a0cc37240 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:14.582977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e1db2-c5a7-4c5e-8541-802a0cc37240 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:14.582977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e1db2-c5a7-4c5e-8541-802a0cc37240 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:14.583430 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e1db2-c5a7-4c5e-8541-802a0cc37240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:14.583708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e1db2-c5a7-4c5e-8541-802a0cc37240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:14.585141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e1db2-c5a7-4c5e-8541-802a0cc37240 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:14.597716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 552/1082] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:14.646739 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560b8f-9667-4112-88c6-7b3c708f1356 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:14.687222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc214dd-7922-44b5-aba3-c46cbcc6de9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:14.687870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc214dd-7922-44b5-aba3-c46cbcc6de9a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:14.688142 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc214dd-7922-44b5-aba3-c46cbcc6de9a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:14.688408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc214dd-7922-44b5-aba3-c46cbcc6de9a None admin] PATH_INFO: `/v3/projects/3b77605fc7624a358a0cfed28e4c55a7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:14.689591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc214dd-7922-44b5-aba3-c46cbcc6de9a None admin] RBAC: Authorizing `identity:get_project(project_id=3b77605fc7624a358a0cfed28e4c55a7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:14.691057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc214dd-7922-44b5-aba3-c46cbcc6de9a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:14.692908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 531/1083] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:14:14 2022] GET /identity/v3/projects/3b77605fc7624a358a0cfed28e4c55a7 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:14.699220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e1db2-c5a7-4c5e-8541-802a0cc37240 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:14.816208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e1db2-c5a7-4c5e-8541-802a0cc37240 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:14.854698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291f1fe-cd4f-495a-b660-51d6ef327eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:14.855365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291f1fe-cd4f-495a-b660-51d6ef327eac None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:14.855642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291f1fe-cd4f-495a-b660-51d6ef327eac None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:14.855907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291f1fe-cd4f-495a-b660-51d6ef327eac None admin] PATH_INFO: `/v3/projects/3b77605fc7624a358a0cfed28e4c55a7/users/fdf9ce5d064148e78b75230e46a01759/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:14.858149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291f1fe-cd4f-495a-b660-51d6ef327eac None admin] RBAC: Authorizing `identity:create_grant(project_id=3b77605fc7624a358a0cfed28e4c55a7, user_id=fdf9ce5d064148e78b75230e46a01759, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:14.859580 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291f1fe-cd4f-495a-b660-51d6ef327eac None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:14.937213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 553/1084] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:14:14 2022] PUT /identity/v3/projects/3b77605fc7624a358a0cfed28e4c55a7/users/fdf9ce5d064148e78b75230e46a01759/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:14:16.347170 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167c009-5764-4e70-b801-5fe50381d393 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.347705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167c009-5764-4e70-b801-5fe50381d393 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.347705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167c009-5764-4e70-b801-5fe50381d393 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.349037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 532/1085] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:16.354183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2299d-4722-4a62-b229-032a617b73f4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.354595 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2299d-4722-4a62-b229-032a617b73f4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.354860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2299d-4722-4a62-b229-032a617b73f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.374329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-33e2299d-4722-4a62-b229-032a617b73f4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:16.375499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-33e2299d-4722-4a62-b229-032a617b73f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:16.416935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:16.442925 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 554/1086] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:16.712961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167c009-5764-4e70-b801-5fe50381d393 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:16.744305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:16.816304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167c009-5764-4e70-b801-5fe50381d393 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:16.850302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331ff32-3e7f-44f4-9422-c24da71601b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:16.850919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331ff32-3e7f-44f4-9422-c24da71601b3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.851191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331ff32-3e7f-44f4-9422-c24da71601b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.851457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331ff32-3e7f-44f4-9422-c24da71601b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.858681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf4621-adc1-4e05-a78f-708691ba2966 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.858967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf4621-adc1-4e05-a78f-708691ba2966 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.859247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf4621-adc1-4e05-a78f-708691ba2966 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.860582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 555/1087] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:16.861984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331ff32-3e7f-44f4-9422-c24da71601b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:16.865361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdca5fb-43df-4ccc-9737-d005d00b9f84 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.865678 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdca5fb-43df-4ccc-9737-d005d00b9f84 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.865952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdca5fb-43df-4ccc-9737-d005d00b9f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-f331ff32-3e7f-44f4-9422-c24da71601b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:16.866709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:16.867783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:16.868684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 533/1088] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 1 05:14:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:16.874170 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331ff32-3e7f-44f4-9422-c24da71601b3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:16.884190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6cdca5fb-43df-4ccc-9737-d005d00b9f84 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:16.885374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6cdca5fb-43df-4ccc-9737-d005d00b9f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:16.952228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 556/1089] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:14:16.963824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:16.973939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331ff32-3e7f-44f4-9422-c24da71601b3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.007199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500d3f1-6fee-44d1-8dbc-09d130609e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.007792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500d3f1-6fee-44d1-8dbc-09d130609e6e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.008064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500d3f1-6fee-44d1-8dbc-09d130609e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.008329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500d3f1-6fee-44d1-8dbc-09d130609e6e None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.009004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500d3f1-6fee-44d1-8dbc-09d130609e6e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:17.009303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500d3f1-6fee-44d1-8dbc-09d130609e6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.010684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500d3f1-6fee-44d1-8dbc-09d130609e6e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.018454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 534/1090] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 1 05:14:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:17.024083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdca5fb-43df-4ccc-9737-d005d00b9f84 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.122394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdca5fb-43df-4ccc-9737-d005d00b9f84 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.129607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.158001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a596fcb-1bda-4652-9523-0dc033668803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.158629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a596fcb-1bda-4652-9523-0dc033668803 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.158909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a596fcb-1bda-4652-9523-0dc033668803 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.159189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a596fcb-1bda-4652-9523-0dc033668803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.168998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a596fcb-1bda-4652-9523-0dc033668803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.170463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a596fcb-1bda-4652-9523-0dc033668803 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3a596fcb-1bda-4652-9523-0dc033668803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:17.184881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:17.186045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:17.186978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:17.186978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:17.186978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:17.186978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 557/1091] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:17.191853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500d3f1-6fee-44d1-8dbc-09d130609e6e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.227781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a596fcb-1bda-4652-9523-0dc033668803 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.349877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500d3f1-6fee-44d1-8dbc-09d130609e6e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.382514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a596fcb-1bda-4652-9523-0dc033668803 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.384288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdb298-4f20-4a7c-a28f-0f7d5755a6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.384892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdb298-4f20-4a7c-a28f-0f7d5755a6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.385163 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdb298-4f20-4a7c-a28f-0f7d5755a6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.385434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdb298-4f20-4a7c-a28f-0f7d5755a6e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.386112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdb298-4f20-4a7c-a28f-0f7d5755a6e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:17.386410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdb298-4f20-4a7c-a28f-0f7d5755a6e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.388574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdb298-4f20-4a7c-a28f-0f7d5755a6e4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.400157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 535/1092] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:17.406227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdb298-4f20-4a7c-a28f-0f7d5755a6e4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.417239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218650d6-fbdd-4d35-84db-06a118432923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.417910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218650d6-fbdd-4d35-84db-06a118432923 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.418183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218650d6-fbdd-4d35-84db-06a118432923 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.418451 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218650d6-fbdd-4d35-84db-06a118432923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.421190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.429796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218650d6-fbdd-4d35-84db-06a118432923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-218650d6-fbdd-4d35-84db-06a118432923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:17.435288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:17.436333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:17.437205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:17.438434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 558/1093] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:14:17.442529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218650d6-fbdd-4d35-84db-06a118432923 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.461377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.509320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdb298-4f20-4a7c-a28f-0f7d5755a6e4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.543169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218650d6-fbdd-4d35-84db-06a118432923 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.544173 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab70a9-eb6b-4adf-bba9-b541b068b7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.544817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab70a9-eb6b-4adf-bba9-b541b068b7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.545092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab70a9-eb6b-4adf-bba9-b541b068b7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.545368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab70a9-eb6b-4adf-bba9-b541b068b7e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.552332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab70a9-eb6b-4adf-bba9-b541b068b7e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.554093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab70a9-eb6b-4adf-bba9-b541b068b7e0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d9ab70a9-eb6b-4adf-bba9-b541b068b7e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:14:17.565957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:17.566945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:17.567908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 536/1094] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:14:17.573065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab70a9-eb6b-4adf-bba9-b541b068b7e0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.579233 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b74c64-0062-4ef8-95db-c14cf3384165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.579836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b74c64-0062-4ef8-95db-c14cf3384165 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.580110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b74c64-0062-4ef8-95db-c14cf3384165 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.580374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b74c64-0062-4ef8-95db-c14cf3384165 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.581027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b74c64-0062-4ef8-95db-c14cf3384165 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:17.581323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b74c64-0062-4ef8-95db-c14cf3384165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.583113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b74c64-0062-4ef8-95db-c14cf3384165 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.593815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 559/1095] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:17.599449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b74c64-0062-4ef8-95db-c14cf3384165 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.643353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.673336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab70a9-eb6b-4adf-bba9-b541b068b7e0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.679713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.699135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b74c64-0062-4ef8-95db-c14cf3384165 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.708337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4c256-7395-4674-8a57-df17c99f2ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.708972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4c256-7395-4674-8a57-df17c99f2ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.709253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4c256-7395-4674-8a57-df17c99f2ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.709551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4c256-7395-4674-8a57-df17c99f2ed8 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.710274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4c256-7395-4674-8a57-df17c99f2ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:17.710577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4c256-7395-4674-8a57-df17c99f2ed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.712012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4c256-7395-4674-8a57-df17c99f2ed8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.731236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 537/1096] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:17.736870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4c256-7395-4674-8a57-df17c99f2ed8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.737284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5366193-ed0d-4875-a46a-bff562b97328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.737955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366193-ed0d-4875-a46a-bff562b97328 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.738237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366193-ed0d-4875-a46a-bff562b97328 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.738508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366193-ed0d-4875-a46a-bff562b97328 None admin] PATH_INFO: `/v3/users/fdf9ce5d064148e78b75230e46a01759` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.739646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5366193-ed0d-4875-a46a-bff562b97328 None admin] RBAC: Authorizing `identity:get_user(user_id=fdf9ce5d064148e78b75230e46a01759)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.741070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5366193-ed0d-4875-a46a-bff562b97328 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.747995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 560/1097] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/users/fdf9ce5d064148e78b75230e46a01759 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:17.749243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5366193-ed0d-4875-a46a-bff562b97328 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.836450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4c256-7395-4674-8a57-df17c99f2ed8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.850009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5366193-ed0d-4875-a46a-bff562b97328 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.865998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.872377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e369ec6-e227-49c5-b7bb-81b4514c7175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.874106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e369ec6-e227-49c5-b7bb-81b4514c7175 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.874106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e369ec6-e227-49c5-b7bb-81b4514c7175 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.874106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e369ec6-e227-49c5-b7bb-81b4514c7175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.885023 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7380b-c8f9-404c-bc87-fa019267d91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.885660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7380b-c8f9-404c-bc87-fa019267d91f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.885938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7380b-c8f9-404c-bc87-fa019267d91f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.886206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7380b-c8f9-404c-bc87-fa019267d91f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.904201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.942622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e369ec6-e227-49c5-b7bb-81b4514c7175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.944086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e369ec6-e227-49c5-b7bb-81b4514c7175 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.953731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7380b-c8f9-404c-bc87-fa019267d91f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.955162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7380b-c8f9-404c-bc87-fa019267d91f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:18.041886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 538/1098] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 05:14:18.048467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e369ec6-e227-49c5-b7bb-81b4514c7175 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:18.057519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 561/1099] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:14:18.058709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7380b-c8f9-404c-bc87-fa019267d91f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:18.080658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:18.107849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:18.155272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e369ec6-e227-49c5-b7bb-81b4514c7175 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:18.159254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7380b-c8f9-404c-bc87-fa019267d91f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:18.189564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71344d30-0f87-453c-b28f-14b050abb99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:18.190208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71344d30-0f87-453c-b28f-14b050abb99a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:18.193973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71344d30-0f87-453c-b28f-14b050abb99a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:18.193973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71344d30-0f87-453c-b28f-14b050abb99a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:18.194146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30474a6-8879-4e39-b0e2-046e67ece340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:18.194146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30474a6-8879-4e39-b0e2-046e67ece340 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:18.194146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30474a6-8879-4e39-b0e2-046e67ece340 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:18.194146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30474a6-8879-4e39-b0e2-046e67ece340 None admin] PATH_INFO: `/v3/projects/3b77605fc7624a358a0cfed28e4c55a7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:18.195102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30474a6-8879-4e39-b0e2-046e67ece340 None admin] RBAC: Authorizing `identity:get_project(project_id=3b77605fc7624a358a0cfed28e4c55a7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:18.196534 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30474a6-8879-4e39-b0e2-046e67ece340 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:18.198498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 562/1100] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:14:18 2022] GET /identity/v3/projects/3b77605fc7624a358a0cfed28e4c55a7 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:18.201953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71344d30-0f87-453c-b28f-14b050abb99a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:18.203400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71344d30-0f87-453c-b28f-14b050abb99a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:18.205064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30474a6-8879-4e39-b0e2-046e67ece340 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-71344d30-0f87-453c-b28f-14b050abb99a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:18.220930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:18.222192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:18.223777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:18.223777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:18.223777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:18.223777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 539/1101] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:14:18.227747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71344d30-0f87-453c-b28f-14b050abb99a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:18.305906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30474a6-8879-4e39-b0e2-046e67ece340 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:18.311412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:18.323043 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:18.330465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71344d30-0f87-453c-b28f-14b050abb99a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:18.341424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bfd57-7ec8-4eaa-9ee0-fbc867069892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:18.342186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bfd57-7ec8-4eaa-9ee0-fbc867069892 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:18.342466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bfd57-7ec8-4eaa-9ee0-fbc867069892 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:18.342741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bfd57-7ec8-4eaa-9ee0-fbc867069892 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:18.343584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1bfd57-7ec8-4eaa-9ee0-fbc867069892 None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=3b77605fc7624a358a0cfed28e4c55a7, user.id=fdf9ce5d064148e78b75230e46a01759) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:18.343892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1bfd57-7ec8-4eaa-9ee0-fbc867069892 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:18.345313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1bfd57-7ec8-4eaa-9ee0-fbc867069892 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:18.351637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 563/1102] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:18 2022] GET /identity/v3/role_assignments?user.id=fdf9ce5d064148e78b75230e46a01759&scope.project.id=3b77605fc7624a358a0cfed28e4c55a7&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:18.366324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac838a-273d-4fb8-89c5-ae326097b376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:18.366901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac838a-273d-4fb8-89c5-ae326097b376 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:18.367131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac838a-273d-4fb8-89c5-ae326097b376 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:18.367400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac838a-273d-4fb8-89c5-ae326097b376 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:18.368057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac838a-273d-4fb8-89c5-ae326097b376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:18.368362 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac838a-273d-4fb8-89c5-ae326097b376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:18.369837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac838a-273d-4fb8-89c5-ae326097b376 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:18.389385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 540/1103] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:18.390670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bfd57-7ec8-4eaa-9ee0-fbc867069892 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:18.500358 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bfd57-7ec8-4eaa-9ee0-fbc867069892 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:18.534512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a425d9-4e78-4258-b805-ceaf56b5d91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:18.535144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a425d9-4e78-4258-b805-ceaf56b5d91f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:18.535419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a425d9-4e78-4258-b805-ceaf56b5d91f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:18.535692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a425d9-4e78-4258-b805-ceaf56b5d91f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:18.542309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a425d9-4e78-4258-b805-ceaf56b5d91f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:18.543773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a425d9-4e78-4258-b805-ceaf56b5d91f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-11a425d9-4e78-4258-b805-ceaf56b5d91f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:14:18.554914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:18.555829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:18.556746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:14:18.556746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:14:18.556746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:14:18.556746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:14:18.556746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:14:18.556746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:14:18.556746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:18.556746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:18.556746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 564/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:18.562735 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac838a-273d-4fb8-89c5-ae326097b376 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:18.670778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac838a-273d-4fb8-89c5-ae326097b376 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:18.704599 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afbe11-72a7-4455-9a28-9d1bde9c0f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:18.705192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afbe11-72a7-4455-9a28-9d1bde9c0f17 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:18.705478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afbe11-72a7-4455-9a28-9d1bde9c0f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:18.705754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afbe11-72a7-4455-9a28-9d1bde9c0f17 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:18.706430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14afbe11-72a7-4455-9a28-9d1bde9c0f17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:18.706730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14afbe11-72a7-4455-9a28-9d1bde9c0f17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:18.708138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14afbe11-72a7-4455-9a28-9d1bde9c0f17 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:18.726198 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 541/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:14:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:18.727621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a425d9-4e78-4258-b805-ceaf56b5d91f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:18.769364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:18.827441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a425d9-4e78-4258-b805-ceaf56b5d91f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:18.861209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c9086-3c2b-47ff-969f-9d45cbb50a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:18.861846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c9086-3c2b-47ff-969f-9d45cbb50a0b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:18.862126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c9086-3c2b-47ff-969f-9d45cbb50a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:18.862396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c9086-3c2b-47ff-969f-9d45cbb50a0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:18.863121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c9086-3c2b-47ff-969f-9d45cbb50a0b None admin] RBAC: Adding query filter params (role.id=1830bfc851a04ab3a87c51c2a2c923fb, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=7cda3c1f67e54ee08abc014399b07cc5) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:18.863427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c9086-3c2b-47ff-969f-9d45cbb50a0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:18.864848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c9086-3c2b-47ff-969f-9d45cbb50a0b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:18.870594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 565/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:18 2022] GET /identity/v3/role_assignments?user.id=7cda3c1f67e54ee08abc014399b07cc5&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=1830bfc851a04ab3a87c51c2a2c923fb => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:20.295588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90924bbb-af9e-4863-bf6a-699d7c422b3b None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:20.296121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90924bbb-af9e-4863-bf6a-699d7c422b3b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:20.296121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90924bbb-af9e-4863-bf6a-699d7c422b3b None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:20.297480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 542/1107] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:14:20.302591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aab5e9-f50f-47a5-8836-2f879ad1b05c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:20.302966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aab5e9-f50f-47a5-8836-2f879ad1b05c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:20.303124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aab5e9-f50f-47a5-8836-2f879ad1b05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:20.321288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-39aab5e9-f50f-47a5-8836-2f879ad1b05c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:20.322755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-39aab5e9-f50f-47a5-8836-2f879ad1b05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:20.388054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 566/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:20.654437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90924bbb-af9e-4863-bf6a-699d7c422b3b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:20.657382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:20.755579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90924bbb-af9e-4863-bf6a-699d7c422b3b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:20.788334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b176a-b7cd-4a01-8e3f-c686b298bb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:20.788950 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b176a-b7cd-4a01-8e3f-c686b298bb55 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:20.789223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b176a-b7cd-4a01-8e3f-c686b298bb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:20.789633 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b176a-b7cd-4a01-8e3f-c686b298bb55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:20.790619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b176a-b7cd-4a01-8e3f-c686b298bb55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:20.792041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b176a-b7cd-4a01-8e3f-c686b298bb55 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:20.794173 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 543/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:20.799984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aab5e9-f50f-47a5-8836-2f879ad1b05c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:20.832365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:20.896323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aab5e9-f50f-47a5-8836-2f879ad1b05c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:20.928811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d1810-47f2-4598-b1d0-8feefe9d80b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:20.929342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d1810-47f2-4598-b1d0-8feefe9d80b1 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:20.929641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d1810-47f2-4598-b1d0-8feefe9d80b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:20.929906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d1810-47f2-4598-b1d0-8feefe9d80b1 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:20.930549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d1810-47f2-4598-b1d0-8feefe9d80b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:20.931940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d1810-47f2-4598-b1d0-8feefe9d80b1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:20.993523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 567/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:14:20 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:14:22.903775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bceb2b-6ae3-4960-9b2d-ede54b7b9169 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:22.904334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bceb2b-6ae3-4960-9b2d-ede54b7b9169 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:22.904334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bceb2b-6ae3-4960-9b2d-ede54b7b9169 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:22.905686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 544/1111] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:14:22.910843 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7964a2-3ae5-4c26-87f1-453d6cbcdaac None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:22.911261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7964a2-3ae5-4c26-87f1-453d6cbcdaac None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:22.911377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7964a2-3ae5-4c26-87f1-453d6cbcdaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:22.930007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0a7964a2-3ae5-4c26-87f1-453d6cbcdaac None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:22.931191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0a7964a2-3ae5-4c26-87f1-453d6cbcdaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:22.995719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 568/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:23.262938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bceb2b-6ae3-4960-9b2d-ede54b7b9169 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:23.332704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:23.363174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bceb2b-6ae3-4960-9b2d-ede54b7b9169 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:23.397725 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3144c2-7657-4963-81fc-a52183f4f6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:23.398341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3144c2-7657-4963-81fc-a52183f4f6cb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:23.398618 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3144c2-7657-4963-81fc-a52183f4f6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:23.398887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3144c2-7657-4963-81fc-a52183f4f6cb None admin] PATH_INFO: `/v3/roles/acfe3eeb91964f62abb1d8a84c36dccc` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:23.400254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3144c2-7657-4963-81fc-a52183f4f6cb None admin] RBAC: Authorizing `identity:get_role(role_id=acfe3eeb91964f62abb1d8a84c36dccc)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:23.401650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3144c2-7657-4963-81fc-a52183f4f6cb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:23.403516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 545/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:23 2022] GET /identity/v3/roles/acfe3eeb91964f62abb1d8a84c36dccc => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:23.409249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7964a2-3ae5-4c26-87f1-453d6cbcdaac None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:23.429643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:23.506796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7964a2-3ae5-4c26-87f1-453d6cbcdaac None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:23.540256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b905ae-8c10-418d-9583-8619aa2bc292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:23.540820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b905ae-8c10-418d-9583-8619aa2bc292 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:23.541096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b905ae-8c10-418d-9583-8619aa2bc292 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:23.541374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b905ae-8c10-418d-9583-8619aa2bc292 None admin] PATH_INFO: `/v3/users/729968cc0a634676a19528bfa33b8c05` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:23.554094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b905ae-8c10-418d-9583-8619aa2bc292 None admin] RBAC: Authorizing `identity:get_user(user_id=729968cc0a634676a19528bfa33b8c05)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:23.555523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b905ae-8c10-418d-9583-8619aa2bc292 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:23.557095 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 569/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:23 2022] GET /identity/v3/users/729968cc0a634676a19528bfa33b8c05 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:23.563396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3144c2-7657-4963-81fc-a52183f4f6cb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:23.660988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3144c2-7657-4963-81fc-a52183f4f6cb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:23.694055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a92989-73b4-40cb-9628-976bf0dfffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:23.694653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a92989-73b4-40cb-9628-976bf0dfffde None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:23.694929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a92989-73b4-40cb-9628-976bf0dfffde None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:23.695198 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a92989-73b4-40cb-9628-976bf0dfffde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:23.704092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:23.762568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a92989-73b4-40cb-9628-976bf0dfffde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:23.764009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a92989-73b4-40cb-9628-976bf0dfffde None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:23.860981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 546/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:14:23.866897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b905ae-8c10-418d-9583-8619aa2bc292 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:23.943373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:23.964137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b905ae-8c10-418d-9583-8619aa2bc292 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:23.998468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d3847-4c51-4fe6-bdbb-584b6630cf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:23.999083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d3847-4c51-4fe6-bdbb-584b6630cf74 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:23.999356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d3847-4c51-4fe6-bdbb-584b6630cf74 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:23.999625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d3847-4c51-4fe6-bdbb-584b6630cf74 None admin] PATH_INFO: `/v3/projects/3b77605fc7624a358a0cfed28e4c55a7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:24.000717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d3847-4c51-4fe6-bdbb-584b6630cf74 None admin] RBAC: Authorizing `identity:get_project(project_id=3b77605fc7624a358a0cfed28e4c55a7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:24.002154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d3847-4c51-4fe6-bdbb-584b6630cf74 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:24.003743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 570/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:14:23 2022] GET /identity/v3/projects/3b77605fc7624a358a0cfed28e4c55a7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:24.009870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a92989-73b4-40cb-9628-976bf0dfffde None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:24.048253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:24.107421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a92989-73b4-40cb-9628-976bf0dfffde None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:24.141476 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cfc2c-ac37-4cb6-93d1-bf1e24e55adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:24.142016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cfc2c-ac37-4cb6-93d1-bf1e24e55adf None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:24.142290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cfc2c-ac37-4cb6-93d1-bf1e24e55adf None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:24.142557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cfc2c-ac37-4cb6-93d1-bf1e24e55adf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:24.143265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cfc2c-ac37-4cb6-93d1-bf1e24e55adf None admin] RBAC: Adding query filter params (role.id=acfe3eeb91964f62abb1d8a84c36dccc, scope.project.id=3b77605fc7624a358a0cfed28e4c55a7, user.id=729968cc0a634676a19528bfa33b8c05) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:24.143568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cfc2c-ac37-4cb6-93d1-bf1e24e55adf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:24.144999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cfc2c-ac37-4cb6-93d1-bf1e24e55adf None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:24.150247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 547/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:24 2022] GET /identity/v3/role_assignments?user.id=729968cc0a634676a19528bfa33b8c05&scope.project.id=3b77605fc7624a358a0cfed28e4c55a7&role.id=acfe3eeb91964f62abb1d8a84c36dccc => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:26.049962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b316d-f3a6-4e1f-8f7c-0865c4421ffb None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:26.050560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b316d-f3a6-4e1f-8f7c-0865c4421ffb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:26.050560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b316d-f3a6-4e1f-8f7c-0865c4421ffb None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:26.051741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 571/1118] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:26.056963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e87896-6d01-4576-9630-e91aa413d63b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:26.057387 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e87896-6d01-4576-9630-e91aa413d63b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:26.057536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e87896-6d01-4576-9630-e91aa413d63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:26.075645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d4e87896-6d01-4576-9630-e91aa413d63b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:26.076827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d4e87896-6d01-4576-9630-e91aa413d63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:26.143244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 548/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:26.418804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b316d-f3a6-4e1f-8f7c-0865c4421ffb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:26.519034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b316d-f3a6-4e1f-8f7c-0865c4421ffb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:26.551610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7523ccc-21a1-440a-82a3-bd50f9b739d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:26.552228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7523ccc-21a1-440a-82a3-bd50f9b739d8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:26.552507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7523ccc-21a1-440a-82a3-bd50f9b739d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:26.552778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7523ccc-21a1-440a-82a3-bd50f9b739d8 None admin] PATH_INFO: `/v3/roles/acfe3eeb91964f62abb1d8a84c36dccc` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:26.554179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7523ccc-21a1-440a-82a3-bd50f9b739d8 None admin] RBAC: Authorizing `identity:get_role(role_id=acfe3eeb91964f62abb1d8a84c36dccc)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:26.555524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7523ccc-21a1-440a-82a3-bd50f9b739d8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:26.556815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 572/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:26 2022] GET /identity/v3/roles/acfe3eeb91964f62abb1d8a84c36dccc => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:26.563260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e87896-6d01-4576-9630-e91aa413d63b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:26.593839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:26.664951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e87896-6d01-4576-9630-e91aa413d63b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:26.699760 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f7b15-b7c8-466d-be80-40468b4f44a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:26.700281 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f7b15-b7c8-466d-be80-40468b4f44a1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:26.700561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f7b15-b7c8-466d-be80-40468b4f44a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:26.700835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f7b15-b7c8-466d-be80-40468b4f44a1 None admin] PATH_INFO: `/v3/users/729968cc0a634676a19528bfa33b8c05` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:26.701985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f7b15-b7c8-466d-be80-40468b4f44a1 None admin] RBAC: Authorizing `identity:get_user(user_id=729968cc0a634676a19528bfa33b8c05)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:26.703432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f7b15-b7c8-466d-be80-40468b4f44a1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:26.705017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 549/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:26 2022] GET /identity/v3/users/729968cc0a634676a19528bfa33b8c05 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:26.710838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7523ccc-21a1-440a-82a3-bd50f9b739d8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:26.726621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:26.809560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7523ccc-21a1-440a-82a3-bd50f9b739d8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:26.842902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb08862-6dc2-4f04-ac6d-6a455889c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:26.843530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb08862-6dc2-4f04-ac6d-6a455889c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:26.843809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb08862-6dc2-4f04-ac6d-6a455889c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:26.844082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb08862-6dc2-4f04-ac6d-6a455889c5b9 None admin] PATH_INFO: `/v3/projects/3b77605fc7624a358a0cfed28e4c55a7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:26.845196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb08862-6dc2-4f04-ac6d-6a455889c5b9 None admin] RBAC: Authorizing `identity:get_project(project_id=3b77605fc7624a358a0cfed28e4c55a7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:26.846676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb08862-6dc2-4f04-ac6d-6a455889c5b9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:26.848286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 573/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:14:26 2022] GET /identity/v3/projects/3b77605fc7624a358a0cfed28e4c55a7 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:26.854318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f7b15-b7c8-466d-be80-40468b4f44a1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:26.947736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:26.951464 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f7b15-b7c8-466d-be80-40468b4f44a1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:26.986520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b64f7-9b55-4c40-8d66-a52379235f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:26.987202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b64f7-9b55-4c40-8d66-a52379235f4a None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:26.987477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b64f7-9b55-4c40-8d66-a52379235f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:26.987747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b64f7-9b55-4c40-8d66-a52379235f4a None admin] PATH_INFO: `/v3/projects/3b77605fc7624a358a0cfed28e4c55a7/users/729968cc0a634676a19528bfa33b8c05/roles/acfe3eeb91964f62abb1d8a84c36dccc` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:26.989881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b64f7-9b55-4c40-8d66-a52379235f4a None admin] RBAC: Authorizing `identity:create_grant(project_id=3b77605fc7624a358a0cfed28e4c55a7, user_id=729968cc0a634676a19528bfa33b8c05, role_id=acfe3eeb91964f62abb1d8a84c36dccc)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:26.991316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b64f7-9b55-4c40-8d66-a52379235f4a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:27.067508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 550/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:14:26 2022] PUT /identity/v3/projects/3b77605fc7624a358a0cfed28e4c55a7/users/729968cc0a634676a19528bfa33b8c05/roles/acfe3eeb91964f62abb1d8a84c36dccc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:14:28.951678 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f96cac-3207-446f-8eda-cee79d020fa1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:28.952234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f96cac-3207-446f-8eda-cee79d020fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:28.952234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f96cac-3207-446f-8eda-cee79d020fa1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:28.953573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 574/1124] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:28.958834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731e52-ea25-4510-9bfb-9b09e297030d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:28.959311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731e52-ea25-4510-9bfb-9b09e297030d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:28.959417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91731e52-ea25-4510-9bfb-9b09e297030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:28.978398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-91731e52-ea25-4510-9bfb-9b09e297030d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:28.979510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-91731e52-ea25-4510-9bfb-9b09e297030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:29.051471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 551/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:29.326077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f96cac-3207-446f-8eda-cee79d020fa1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:29.407831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:29.426072 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f96cac-3207-446f-8eda-cee79d020fa1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:29.459182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183cc53-f05a-41d4-9c42-29e8a956e3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:29.459801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183cc53-f05a-41d4-9c42-29e8a956e3eb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:29.460080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183cc53-f05a-41d4-9c42-29e8a956e3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:29.460350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183cc53-f05a-41d4-9c42-29e8a956e3eb None admin] PATH_INFO: `/v3/roles/acfe3eeb91964f62abb1d8a84c36dccc` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:29.461758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183cc53-f05a-41d4-9c42-29e8a956e3eb None admin] RBAC: Authorizing `identity:get_role(role_id=acfe3eeb91964f62abb1d8a84c36dccc)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:29.463110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183cc53-f05a-41d4-9c42-29e8a956e3eb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:29.464404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 575/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:29 2022] GET /identity/v3/roles/acfe3eeb91964f62abb1d8a84c36dccc => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:29.470903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91731e52-ea25-4510-9bfb-9b09e297030d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:29.581186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:29.581186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91731e52-ea25-4510-9bfb-9b09e297030d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:29.608090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba75b-c6d6-4975-a56f-b184144e35d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:29.608708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba75b-c6d6-4975-a56f-b184144e35d1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:29.608984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba75b-c6d6-4975-a56f-b184144e35d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:29.609253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba75b-c6d6-4975-a56f-b184144e35d1 None admin] PATH_INFO: `/v3/users/729968cc0a634676a19528bfa33b8c05` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:29.610605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ba75b-c6d6-4975-a56f-b184144e35d1 None admin] RBAC: Authorizing `identity:get_user(user_id=729968cc0a634676a19528bfa33b8c05)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:29.612325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ba75b-c6d6-4975-a56f-b184144e35d1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:29.614047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 552/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:29 2022] GET /identity/v3/users/729968cc0a634676a19528bfa33b8c05 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:29.620161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183cc53-f05a-41d4-9c42-29e8a956e3eb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:29.719697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183cc53-f05a-41d4-9c42-29e8a956e3eb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:29.752997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baad538-1acc-4344-b076-66ad8c040109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:29.753631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baad538-1acc-4344-b076-66ad8c040109 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:29.753909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baad538-1acc-4344-b076-66ad8c040109 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:29.754178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baad538-1acc-4344-b076-66ad8c040109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:29.773140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:29.820380 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baad538-1acc-4344-b076-66ad8c040109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:29.822050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baad538-1acc-4344-b076-66ad8c040109 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:29.916964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 576/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 05:14:29.923545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba75b-c6d6-4975-a56f-b184144e35d1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:30.022071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba75b-c6d6-4975-a56f-b184144e35d1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:30.055169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26bbf6-b236-468d-b71d-605cca28c60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:30.055776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26bbf6-b236-468d-b71d-605cca28c60d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:30.056053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26bbf6-b236-468d-b71d-605cca28c60d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:30.056323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26bbf6-b236-468d-b71d-605cca28c60d None admin] PATH_INFO: `/v3/projects/3b77605fc7624a358a0cfed28e4c55a7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:30.057418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26bbf6-b236-468d-b71d-605cca28c60d None admin] RBAC: Authorizing `identity:get_project(project_id=3b77605fc7624a358a0cfed28e4c55a7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:30.059527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26bbf6-b236-468d-b71d-605cca28c60d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:30.061119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 553/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:14:29 2022] GET /identity/v3/projects/3b77605fc7624a358a0cfed28e4c55a7 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:30.067177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baad538-1acc-4344-b076-66ad8c040109 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:30.115809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:30.165593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baad538-1acc-4344-b076-66ad8c040109 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:30.198365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a0140-3e54-4da3-bc3b-8237b0a69dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:30.199630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a0140-3e54-4da3-bc3b-8237b0a69dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:30.199916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a0140-3e54-4da3-bc3b-8237b0a69dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:30.200187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a0140-3e54-4da3-bc3b-8237b0a69dc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:30.200908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a0140-3e54-4da3-bc3b-8237b0a69dc1 None admin] RBAC: Adding query filter params (role.id=acfe3eeb91964f62abb1d8a84c36dccc, scope.project.id=3b77605fc7624a358a0cfed28e4c55a7, user.id=729968cc0a634676a19528bfa33b8c05) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:30.201214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a0140-3e54-4da3-bc3b-8237b0a69dc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:30.202661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a0140-3e54-4da3-bc3b-8237b0a69dc1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:30.208930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 577/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:30 2022] GET /identity/v3/role_assignments?user.id=729968cc0a634676a19528bfa33b8c05&scope.project.id=3b77605fc7624a358a0cfed28e4c55a7&role.id=acfe3eeb91964f62abb1d8a84c36dccc => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:32.114028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923ce64-7e17-4dae-ab1a-3d54cc65575c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:32.114573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923ce64-7e17-4dae-ab1a-3d54cc65575c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:32.114573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923ce64-7e17-4dae-ab1a-3d54cc65575c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:32.115947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 554/1131] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:32.121129 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d5734-8bbe-4d16-88ed-565b8c33cb2d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:32.121583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d5734-8bbe-4d16-88ed-565b8c33cb2d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:32.121749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d5734-8bbe-4d16-88ed-565b8c33cb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:32.139967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d10d5734-8bbe-4d16-88ed-565b8c33cb2d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:32.141149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d10d5734-8bbe-4d16-88ed-565b8c33cb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:32.216518 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 578/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:14:32.483648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923ce64-7e17-4dae-ab1a-3d54cc65575c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:32.510878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:32.584934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923ce64-7e17-4dae-ab1a-3d54cc65575c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:32.617091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf6730-748a-4468-bc48-0e74dbb69fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:32.617721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf6730-748a-4468-bc48-0e74dbb69fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:32.617996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf6730-748a-4468-bc48-0e74dbb69fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:32.618264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf6730-748a-4468-bc48-0e74dbb69fe8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:32.619348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf6730-748a-4468-bc48-0e74dbb69fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:32.621290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf6730-748a-4468-bc48-0e74dbb69fe8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:32.622854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 555/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:32.628575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d5734-8bbe-4d16-88ed-565b8c33cb2d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:32.673232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:32.726274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d5734-8bbe-4d16-88ed-565b8c33cb2d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:32.761858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b763e-9aec-4349-bff2-854282ebd48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:32.762407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b763e-9aec-4349-bff2-854282ebd48e None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:32.762686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b763e-9aec-4349-bff2-854282ebd48e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:32.762955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b763e-9aec-4349-bff2-854282ebd48e None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:32.763608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b763e-9aec-4349-bff2-854282ebd48e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:32.765019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b763e-9aec-4349-bff2-854282ebd48e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:32.864036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 579/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:14:32 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:14:34.881786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8ed72-2026-4711-929e-2df4d20e115e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:34.882331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8ed72-2026-4711-929e-2df4d20e115e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:34.882331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8ed72-2026-4711-929e-2df4d20e115e None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:34.883656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 556/1135] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:34.888884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a299b-8a33-484a-a4ba-55deca4f0111 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:34.889321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a299b-8a33-484a-a4ba-55deca4f0111 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:34.889426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a299b-8a33-484a-a4ba-55deca4f0111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:34.907952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-787a299b-8a33-484a-a4ba-55deca4f0111 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:34.909195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-787a299b-8a33-484a-a4ba-55deca4f0111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:34.974183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 580/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:35.246963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8ed72-2026-4711-929e-2df4d20e115e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:35.341696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:35.349117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8ed72-2026-4711-929e-2df4d20e115e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:35.382454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7d99a-11b4-454e-9df5-79024d9c510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:35.383063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d99a-11b4-454e-9df5-79024d9c510a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:35.383339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d99a-11b4-454e-9df5-79024d9c510a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:35.383607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d99a-11b4-454e-9df5-79024d9c510a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:35.384698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7d99a-11b4-454e-9df5-79024d9c510a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:35.386754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7d99a-11b4-454e-9df5-79024d9c510a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:35.388324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 557/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:35.394182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a299b-8a33-484a-a4ba-55deca4f0111 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:35.420714 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:35.491076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a299b-8a33-484a-a4ba-55deca4f0111 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:35.524818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ae30a-6f4f-49fe-8030-89ad61ac3c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:35.525348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ae30a-6f4f-49fe-8030-89ad61ac3c01 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:35.525656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ae30a-6f4f-49fe-8030-89ad61ac3c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:35.525929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ae30a-6f4f-49fe-8030-89ad61ac3c01 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:35.526566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ae30a-6f4f-49fe-8030-89ad61ac3c01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:35.528603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ae30a-6f4f-49fe-8030-89ad61ac3c01 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:35.615997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 581/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:14:35 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:14:37.533735 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6b585-2278-4be8-82ee-e3057baa24c9 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:37.534310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6b585-2278-4be8-82ee-e3057baa24c9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:37.534310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6b585-2278-4be8-82ee-e3057baa24c9 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:37.535600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 558/1139] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:37.540837 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af38993-bb8e-4bcf-82b3-d7c07d21420d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:37.541254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af38993-bb8e-4bcf-82b3-d7c07d21420d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:37.541388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af38993-bb8e-4bcf-82b3-d7c07d21420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:37.560650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5af38993-bb8e-4bcf-82b3-d7c07d21420d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:37.561878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5af38993-bb8e-4bcf-82b3-d7c07d21420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:37.626790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 582/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:37.898735 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6b585-2278-4be8-82ee-e3057baa24c9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:37.997476 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6b585-2278-4be8-82ee-e3057baa24c9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.030921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1ed1c-0850-4e91-81aa-7af52df45841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.031528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1ed1c-0850-4e91-81aa-7af52df45841 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.031802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1ed1c-0850-4e91-81aa-7af52df45841 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.032070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1ed1c-0850-4e91-81aa-7af52df45841 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.041840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1ed1c-0850-4e91-81aa-7af52df45841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-f1d1ed1c-0850-4e91-81aa-7af52df45841 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:38.046827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:38.048065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:38.049365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 559/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:14:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:38.053759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af38993-bb8e-4bcf-82b3-d7c07d21420d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:38.068570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:38.152299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af38993-bb8e-4bcf-82b3-d7c07d21420d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.186002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd718a-21f2-4495-8800-af0817f35d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.186608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd718a-21f2-4495-8800-af0817f35d17 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.186887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd718a-21f2-4495-8800-af0817f35d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.187158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd718a-21f2-4495-8800-af0817f35d17 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.187818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd718a-21f2-4495-8800-af0817f35d17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:38.188122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd718a-21f2-4495-8800-af0817f35d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.189503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd718a-21f2-4495-8800-af0817f35d17 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.197900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 583/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:38.203869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1ed1c-0850-4e91-81aa-7af52df45841 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:38.228863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:38.304278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1ed1c-0850-4e91-81aa-7af52df45841 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.338576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd4a15-15b8-4094-bf2d-a2560ff9f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.339409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd4a15-15b8-4094-bf2d-a2560ff9f7ef None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.339756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd4a15-15b8-4094-bf2d-a2560ff9f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.340069 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd4a15-15b8-4094-bf2d-a2560ff9f7ef None admin] PATH_INFO: `/v3/users/24e9e38d435a472a9829d2a647617892` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.352745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd4a15-15b8-4094-bf2d-a2560ff9f7ef None admin] RBAC: Authorizing `identity:get_user(user_id=24e9e38d435a472a9829d2a647617892)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.354217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd4a15-15b8-4094-bf2d-a2560ff9f7ef None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.356350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 560/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/users/24e9e38d435a472a9829d2a647617892 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:38.362539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd718a-21f2-4495-8800-af0817f35d17 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:38.441121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:38.465992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd718a-21f2-4495-8800-af0817f35d17 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.499318 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7fbcb-90aa-4dde-bd0a-3865c6952fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.499910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7fbcb-90aa-4dde-bd0a-3865c6952fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.500185 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7fbcb-90aa-4dde-bd0a-3865c6952fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.500452 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7fbcb-90aa-4dde-bd0a-3865c6952fc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.565237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7fbcb-90aa-4dde-bd0a-3865c6952fc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.566699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7fbcb-90aa-4dde-bd0a-3865c6952fc4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.638240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:38.662212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 584/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:14:38.668363 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd4a15-15b8-4094-bf2d-a2560ff9f7ef None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:38.748197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:38.767275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd4a15-15b8-4094-bf2d-a2560ff9f7ef None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.800299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcf2a1-2495-4090-9905-ddc0925ef49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.800910 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcf2a1-2495-4090-9905-ddc0925ef49d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.801184 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcf2a1-2495-4090-9905-ddc0925ef49d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.801467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcf2a1-2495-4090-9905-ddc0925ef49d None admin] PATH_INFO: `/v3/projects/31432df392f7452ea398ad4340b616df` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.802579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcf2a1-2495-4090-9905-ddc0925ef49d None admin] RBAC: Authorizing `identity:get_project(project_id=31432df392f7452ea398ad4340b616df)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.804001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcf2a1-2495-4090-9905-ddc0925ef49d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.805638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 561/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/projects/31432df392f7452ea398ad4340b616df => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:38.812604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7fbcb-90aa-4dde-bd0a-3865c6952fc4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:38.909639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7fbcb-90aa-4dde-bd0a-3865c6952fc4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.943128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13364a3e-bead-44aa-b351-888e060e84fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.943717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13364a3e-bead-44aa-b351-888e060e84fe None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.943997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13364a3e-bead-44aa-b351-888e060e84fe None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.944265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13364a3e-bead-44aa-b351-888e060e84fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.944968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13364a3e-bead-44aa-b351-888e060e84fe None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=31432df392f7452ea398ad4340b616df, user.id=24e9e38d435a472a9829d2a647617892) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:38.945270 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13364a3e-bead-44aa-b351-888e060e84fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.946954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13364a3e-bead-44aa-b351-888e060e84fe None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.952352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 585/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/role_assignments?user.id=24e9e38d435a472a9829d2a647617892&scope.project.id=31432df392f7452ea398ad4340b616df&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:40.821229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f719f1-39b9-4bb0-9621-6ad954a7a50a None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:40.821790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f719f1-39b9-4bb0-9621-6ad954a7a50a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:40.821790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f719f1-39b9-4bb0-9621-6ad954a7a50a None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:40.823178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 562/1147] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:40.828413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dfc9d-90b5-49e6-b94e-3d2e3a865fa6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:40.828812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dfc9d-90b5-49e6-b94e-3d2e3a865fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:40.828959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dfc9d-90b5-49e6-b94e-3d2e3a865fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:40.848023 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c91dfc9d-90b5-49e6-b94e-3d2e3a865fa6 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:40.849202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c91dfc9d-90b5-49e6-b94e-3d2e3a865fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:40.878332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:40.914772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 586/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:41.183360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f719f1-39b9-4bb0-9621-6ad954a7a50a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:41.284586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f719f1-39b9-4bb0-9621-6ad954a7a50a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:41.317139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1a411-806c-49c1-aba5-44e2a39aab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:41.317776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1a411-806c-49c1-aba5-44e2a39aab8a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:41.318053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1a411-806c-49c1-aba5-44e2a39aab8a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:41.318318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1a411-806c-49c1-aba5-44e2a39aab8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:41.328895 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1a411-806c-49c1-aba5-44e2a39aab8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-13d1a411-806c-49c1-aba5-44e2a39aab8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:41.333606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:41.334703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:41.335707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 563/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:14:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:41.341708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dfc9d-90b5-49e6-b94e-3d2e3a865fa6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:41.436890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dfc9d-90b5-49e6-b94e-3d2e3a865fa6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:41.469758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb42d4-9d58-45d6-8bff-f3e9e2179a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:41.470353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb42d4-9d58-45d6-8bff-f3e9e2179a5f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:41.470625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb42d4-9d58-45d6-8bff-f3e9e2179a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:41.470895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb42d4-9d58-45d6-8bff-f3e9e2179a5f None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:41.471546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb42d4-9d58-45d6-8bff-f3e9e2179a5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:41.471845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb42d4-9d58-45d6-8bff-f3e9e2179a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:41.473193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb42d4-9d58-45d6-8bff-f3e9e2179a5f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:41.480951 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 587/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:14:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:41.486619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1a411-806c-49c1-aba5-44e2a39aab8a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:41.498512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:41.586600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1a411-806c-49c1-aba5-44e2a39aab8a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:41.619141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f3672-5c08-45c2-b24a-ae530e020f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:41.619747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f3672-5c08-45c2-b24a-ae530e020f38 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:41.620636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f3672-5c08-45c2-b24a-ae530e020f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:41.621139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f3672-5c08-45c2-b24a-ae530e020f38 None admin] PATH_INFO: `/v3/users/24e9e38d435a472a9829d2a647617892` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:41.622297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f3672-5c08-45c2-b24a-ae530e020f38 None admin] RBAC: Authorizing `identity:get_user(user_id=24e9e38d435a472a9829d2a647617892)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:41.623719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f3672-5c08-45c2-b24a-ae530e020f38 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:41.625262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 564/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:41 2022] GET /identity/v3/users/24e9e38d435a472a9829d2a647617892 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:41.631018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb42d4-9d58-45d6-8bff-f3e9e2179a5f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:41.661818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:41.734878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb42d4-9d58-45d6-8bff-f3e9e2179a5f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:41.768083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecc411-7b30-4f3b-8914-c150fc4af3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:41.768706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecc411-7b30-4f3b-8914-c150fc4af3be None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:41.768984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecc411-7b30-4f3b-8914-c150fc4af3be None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:41.769257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecc411-7b30-4f3b-8914-c150fc4af3be None admin] PATH_INFO: `/v3/projects/31432df392f7452ea398ad4340b616df` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:41.770382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecc411-7b30-4f3b-8914-c150fc4af3be None admin] RBAC: Authorizing `identity:get_project(project_id=31432df392f7452ea398ad4340b616df)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:41.771828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecc411-7b30-4f3b-8914-c150fc4af3be None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:41.773433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 588/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:14:41 2022] GET /identity/v3/projects/31432df392f7452ea398ad4340b616df => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:41.779419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f3672-5c08-45c2-b24a-ae530e020f38 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:41.855698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:41.878945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f3672-5c08-45c2-b24a-ae530e020f38 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:41.912070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbef2e-4d17-44c7-93e1-d0d3a82c267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:41.912703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbef2e-4d17-44c7-93e1-d0d3a82c267c None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:41.912978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbef2e-4d17-44c7-93e1-d0d3a82c267c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:41.913244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbef2e-4d17-44c7-93e1-d0d3a82c267c None admin] PATH_INFO: `/v3/projects/31432df392f7452ea398ad4340b616df/users/24e9e38d435a472a9829d2a647617892/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:41.915115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbef2e-4d17-44c7-93e1-d0d3a82c267c None admin] RBAC: Authorizing `identity:create_grant(project_id=31432df392f7452ea398ad4340b616df, user_id=24e9e38d435a472a9829d2a647617892, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:41.916527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbef2e-4d17-44c7-93e1-d0d3a82c267c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:41.974488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 565/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:14:41 2022] PUT /identity/v3/projects/31432df392f7452ea398ad4340b616df/users/24e9e38d435a472a9829d2a647617892/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:14:43.875241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ac79-7ebe-4a2d-beef-e1ee0bbb2143 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:43.875759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ac79-7ebe-4a2d-beef-e1ee0bbb2143 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:43.875759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ac79-7ebe-4a2d-beef-e1ee0bbb2143 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:43.877105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 589/1154] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:43.882247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3504d-e39f-46ce-b54d-890052fdc17e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:43.882666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3504d-e39f-46ce-b54d-890052fdc17e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:43.882772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3504d-e39f-46ce-b54d-890052fdc17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:43.901739 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d9c3504d-e39f-46ce-b54d-890052fdc17e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:43.908336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d9c3504d-e39f-46ce-b54d-890052fdc17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:43.970537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 566/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:44.240558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405ac79-7ebe-4a2d-beef-e1ee0bbb2143 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:44.352767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:44.357682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405ac79-7ebe-4a2d-beef-e1ee0bbb2143 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:44.390877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529a8c-3ca5-4ae9-b37a-ce6a937e9610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:44.391494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529a8c-3ca5-4ae9-b37a-ce6a937e9610 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:44.391771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529a8c-3ca5-4ae9-b37a-ce6a937e9610 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:44.392039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529a8c-3ca5-4ae9-b37a-ce6a937e9610 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:44.402412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37529a8c-3ca5-4ae9-b37a-ce6a937e9610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-37529a8c-3ca5-4ae9-b37a-ce6a937e9610 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:44.407146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:44.408229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:44.409099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 590/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:44.414525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3504d-e39f-46ce-b54d-890052fdc17e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:44.478712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:44.513559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3504d-e39f-46ce-b54d-890052fdc17e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:44.548325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeade40c-d6d3-4b10-9509-153277485d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:44.548914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeade40c-d6d3-4b10-9509-153277485d40 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:44.549186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeade40c-d6d3-4b10-9509-153277485d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:44.549475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeade40c-d6d3-4b10-9509-153277485d40 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:44.550133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeade40c-d6d3-4b10-9509-153277485d40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:44.550428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeade40c-d6d3-4b10-9509-153277485d40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:44.551778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeade40c-d6d3-4b10-9509-153277485d40 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:44.559774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 567/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:44.565648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529a8c-3ca5-4ae9-b37a-ce6a937e9610 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:44.665721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529a8c-3ca5-4ae9-b37a-ce6a937e9610 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:44.699025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1a871-4d18-4ab7-89b0-23b1f47c2329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:44.699644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1a871-4d18-4ab7-89b0-23b1f47c2329 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:44.699924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1a871-4d18-4ab7-89b0-23b1f47c2329 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:44.700195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1a871-4d18-4ab7-89b0-23b1f47c2329 None admin] PATH_INFO: `/v3/users/24e9e38d435a472a9829d2a647617892` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:44.701301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1a871-4d18-4ab7-89b0-23b1f47c2329 None admin] RBAC: Authorizing `identity:get_user(user_id=24e9e38d435a472a9829d2a647617892)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:44.702827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1a871-4d18-4ab7-89b0-23b1f47c2329 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:44.704398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 591/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:44 2022] GET /identity/v3/users/24e9e38d435a472a9829d2a647617892 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:44.710500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeade40c-d6d3-4b10-9509-153277485d40 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:44.807856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeade40c-d6d3-4b10-9509-153277485d40 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:44.832740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:44.840818 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65468350-f2af-46e3-a956-b005d428af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:44.841413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468350-f2af-46e3-a956-b005d428af1e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:44.845424 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468350-f2af-46e3-a956-b005d428af1e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:44.845424 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65468350-f2af-46e3-a956-b005d428af1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:44.908531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65468350-f2af-46e3-a956-b005d428af1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:44.910005 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65468350-f2af-46e3-a956-b005d428af1e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:45.004229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 568/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:14:45.010189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1a871-4d18-4ab7-89b0-23b1f47c2329 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:45.037998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:45.108802 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1a871-4d18-4ab7-89b0-23b1f47c2329 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:45.141566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221a6e5-87cb-4ceb-8b72-a59d99b42865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:45.142183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a6e5-87cb-4ceb-8b72-a59d99b42865 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:45.142457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a6e5-87cb-4ceb-8b72-a59d99b42865 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:45.142724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a6e5-87cb-4ceb-8b72-a59d99b42865 None admin] PATH_INFO: `/v3/projects/31432df392f7452ea398ad4340b616df` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:45.143818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221a6e5-87cb-4ceb-8b72-a59d99b42865 None admin] RBAC: Authorizing `identity:get_project(project_id=31432df392f7452ea398ad4340b616df)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:45.145227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221a6e5-87cb-4ceb-8b72-a59d99b42865 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:45.146840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 592/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:14:45 2022] GET /identity/v3/projects/31432df392f7452ea398ad4340b616df => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:45.153473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65468350-f2af-46e3-a956-b005d428af1e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:45.180501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:45.250970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65468350-f2af-46e3-a956-b005d428af1e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:45.283931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b903c0-abe6-4f3e-bd7a-826bc9474f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:45.284527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b903c0-abe6-4f3e-bd7a-826bc9474f84 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:45.284802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b903c0-abe6-4f3e-bd7a-826bc9474f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:45.285070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b903c0-abe6-4f3e-bd7a-826bc9474f84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:45.285808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b903c0-abe6-4f3e-bd7a-826bc9474f84 None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=31432df392f7452ea398ad4340b616df, user.id=24e9e38d435a472a9829d2a647617892) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:45.286114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b903c0-abe6-4f3e-bd7a-826bc9474f84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:45.287532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b903c0-abe6-4f3e-bd7a-826bc9474f84 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:45.293118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 569/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:45 2022] GET /identity/v3/role_assignments?user.id=24e9e38d435a472a9829d2a647617892&scope.project.id=31432df392f7452ea398ad4340b616df&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:47.205784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b788d2-685d-45bf-93c6-a142ea95b07d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:47.206313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b788d2-685d-45bf-93c6-a142ea95b07d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:47.206313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b788d2-685d-45bf-93c6-a142ea95b07d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:47.207664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 593/1162] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:47.212976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb29eb3-3c02-4ec8-a9c4-30573379cbb4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:47.213441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb29eb3-3c02-4ec8-a9c4-30573379cbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:47.213647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb29eb3-3c02-4ec8-a9c4-30573379cbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:47.231871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ecb29eb3-3c02-4ec8-a9c4-30573379cbb4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:47.233042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ecb29eb3-3c02-4ec8-a9c4-30573379cbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:47.300802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 570/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:47.567739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b788d2-685d-45bf-93c6-a142ea95b07d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:47.660198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:47.667661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b788d2-685d-45bf-93c6-a142ea95b07d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:47.701556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e6bc9-51de-43ed-84d7-7d918bdcab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:47.702156 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e6bc9-51de-43ed-84d7-7d918bdcab05 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:47.702436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e6bc9-51de-43ed-84d7-7d918bdcab05 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:47.702710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e6bc9-51de-43ed-84d7-7d918bdcab05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:47.769517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e6bc9-51de-43ed-84d7-7d918bdcab05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:47.770944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e6bc9-51de-43ed-84d7-7d918bdcab05 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:47.859381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:47.866500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 594/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:14:47.872787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb29eb3-3c02-4ec8-a9c4-30573379cbb4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:47.896681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:47.971269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb29eb3-3c02-4ec8-a9c4-30573379cbb4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:48.003699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d552f-7f67-4317-8d72-18c7d76169e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:48.004315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d552f-7f67-4317-8d72-18c7d76169e9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:48.004589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d552f-7f67-4317-8d72-18c7d76169e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:48.004857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d552f-7f67-4317-8d72-18c7d76169e9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:48.011817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d552f-7f67-4317-8d72-18c7d76169e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:48.013245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d552f-7f67-4317-8d72-18c7d76169e9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-064d552f-7f67-4317-8d72-18c7d76169e9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:48.024137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:48.025318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:14:48.027061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 571/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 1 05:14:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:48.031997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e6bc9-51de-43ed-84d7-7d918bdcab05 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:48.127567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e6bc9-51de-43ed-84d7-7d918bdcab05 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:48.159904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07fb8e-bea3-462b-856e-5e6530e14fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:48.160495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07fb8e-bea3-462b-856e-5e6530e14fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:48.160770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07fb8e-bea3-462b-856e-5e6530e14fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:48.161039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07fb8e-bea3-462b-856e-5e6530e14fb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:48.161690 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07fb8e-bea3-462b-856e-5e6530e14fb9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:48.161990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07fb8e-bea3-462b-856e-5e6530e14fb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:48.163330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07fb8e-bea3-462b-856e-5e6530e14fb9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:48.170709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 595/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 1 05:14:48 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:50.068239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7c1c1-2b23-43e1-9e1f-2e06f50d40a4 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:50.068763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7c1c1-2b23-43e1-9e1f-2e06f50d40a4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:50.068763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7c1c1-2b23-43e1-9e1f-2e06f50d40a4 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:50.070060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 572/1167] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:50.075848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99be43d-ad7f-4b7d-872f-bd48b064326a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:50.076252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99be43d-ad7f-4b7d-872f-bd48b064326a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:50.076417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99be43d-ad7f-4b7d-872f-bd48b064326a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:50.095256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e99be43d-ad7f-4b7d-872f-bd48b064326a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:50.096446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e99be43d-ad7f-4b7d-872f-bd48b064326a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:50.149163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:50.162885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 596/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:50.431585 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7c1c1-2b23-43e1-9e1f-2e06f50d40a4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:50.514815 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:50.533329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7c1c1-2b23-43e1-9e1f-2e06f50d40a4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:50.566943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd775d-9614-482b-a966-210fdebadff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:50.567539 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd775d-9614-482b-a966-210fdebadff9 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:50.567817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd775d-9614-482b-a966-210fdebadff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:50.568088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd775d-9614-482b-a966-210fdebadff9 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:50.568746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd775d-9614-482b-a966-210fdebadff9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:50.570129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd775d-9614-482b-a966-210fdebadff9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:50.627114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 573/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Thu Sep 1 05:14:50 2022] POST /identity/v3/domains => generated 267 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:14:52.539341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49490394-f24f-43bc-8163-8a116c508c38 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:52.539877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49490394-f24f-43bc-8163-8a116c508c38 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:52.539877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49490394-f24f-43bc-8163-8a116c508c38 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:52.541834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 597/1170] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:52.546982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96df53-10b9-496c-9eff-be4130facf0b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:52.547404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96df53-10b9-496c-9eff-be4130facf0b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:52.547530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96df53-10b9-496c-9eff-be4130facf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:52.566050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2c96df53-10b9-496c-9eff-be4130facf0b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:52.567233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2c96df53-10b9-496c-9eff-be4130facf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:52.633569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 574/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:52.899370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49490394-f24f-43bc-8163-8a116c508c38 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:52.999157 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49490394-f24f-43bc-8163-8a116c508c38 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:53.032587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18899519-0de7-4b92-b722-d23f5d62ad25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:53.033197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18899519-0de7-4b92-b722-d23f5d62ad25 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:53.033487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18899519-0de7-4b92-b722-d23f5d62ad25 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:53.033756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18899519-0de7-4b92-b722-d23f5d62ad25 None admin] PATH_INFO: `/v3/domains/4a843f212bbc4d5da66df6bcc1f7bccf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:53.044838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18899519-0de7-4b92-b722-d23f5d62ad25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4a843f212bbc4d5da66df6bcc1f7bccf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:53.046306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18899519-0de7-4b92-b722-d23f5d62ad25 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:53.047854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 598/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Sep 1 05:14:52 2022] GET /identity/v3/domains/4a843f212bbc4d5da66df6bcc1f7bccf => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:53.053643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96df53-10b9-496c-9eff-be4130facf0b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:53.122775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:53.152040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96df53-10b9-496c-9eff-be4130facf0b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:53.185246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9d523-ec42-43ff-8009-434585651b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:53.185863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9d523-ec42-43ff-8009-434585651b86 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:53.186152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9d523-ec42-43ff-8009-434585651b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:53.186420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9d523-ec42-43ff-8009-434585651b86 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:53.187064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9d523-ec42-43ff-8009-434585651b86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:53.188473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9d523-ec42-43ff-8009-434585651b86 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:53.313304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 575/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:14:53 2022] POST /identity/v3/projects => generated 361 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:14:55.232910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094ea26-a73c-45ee-9418-30cd0eb0d4bd None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:55.233559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094ea26-a73c-45ee-9418-30cd0eb0d4bd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:55.233559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094ea26-a73c-45ee-9418-30cd0eb0d4bd None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:55.234872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 599/1174] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:55.240016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7022450-3e73-4e48-b4d7-4d4295fc1001 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:55.240430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7022450-3e73-4e48-b4d7-4d4295fc1001 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:55.240564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7022450-3e73-4e48-b4d7-4d4295fc1001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:55.258788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f7022450-3e73-4e48-b4d7-4d4295fc1001 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:55.260559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f7022450-3e73-4e48-b4d7-4d4295fc1001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:55.324996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 576/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:55.591694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094ea26-a73c-45ee-9418-30cd0eb0d4bd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:55.641677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:55.691940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094ea26-a73c-45ee-9418-30cd0eb0d4bd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:55.724858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab63a0a-e799-436c-a846-aa4f58f3e1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:55.725516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63a0a-e799-436c-a846-aa4f58f3e1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:55.725800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63a0a-e799-436c-a846-aa4f58f3e1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:55.726067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63a0a-e799-436c-a846-aa4f58f3e1b5 None admin] PATH_INFO: `/v3/domains/4a843f212bbc4d5da66df6bcc1f7bccf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:55.727189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab63a0a-e799-436c-a846-aa4f58f3e1b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4a843f212bbc4d5da66df6bcc1f7bccf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:55.729277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab63a0a-e799-436c-a846-aa4f58f3e1b5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:55.730907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 600/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Sep 1 05:14:55 2022] GET /identity/v3/domains/4a843f212bbc4d5da66df6bcc1f7bccf => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:55.736845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7022450-3e73-4e48-b4d7-4d4295fc1001 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:55.792909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:55.835230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7022450-3e73-4e48-b4d7-4d4295fc1001 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:55.869083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf770e-c8ae-4b90-bf7a-14848a02290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:55.869647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf770e-c8ae-4b90-bf7a-14848a02290b None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:55.869919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf770e-c8ae-4b90-bf7a-14848a02290b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:55.870181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf770e-c8ae-4b90-bf7a-14848a02290b None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:55.870812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf770e-c8ae-4b90-bf7a-14848a02290b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:55.872218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf770e-c8ae-4b90-bf7a-14848a02290b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:55.970529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 577/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:14:55 2022] POST /identity/v3/users => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:14:57.928944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe286637-c90d-40d3-9bbd-52794a93f5e5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:57.929558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe286637-c90d-40d3-9bbd-52794a93f5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:57.929558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe286637-c90d-40d3-9bbd-52794a93f5e5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:57.930844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 601/1178] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:57.936662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3d12e-997b-44d6-84fd-fdbb1ed3cd91 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:57.937101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3d12e-997b-44d6-84fd-fdbb1ed3cd91 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:57.937200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3d12e-997b-44d6-84fd-fdbb1ed3cd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:57.955358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d3c3d12e-997b-44d6-84fd-fdbb1ed3cd91 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:57.956535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d3c3d12e-997b-44d6-84fd-fdbb1ed3cd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:58.022428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 578/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:58.313900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe286637-c90d-40d3-9bbd-52794a93f5e5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:58.413293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe286637-c90d-40d3-9bbd-52794a93f5e5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:58.429072 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:58.446300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad333-e2d1-4da4-b2fc-867272d1a1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:58.447556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad333-e2d1-4da4-b2fc-867272d1a1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:58.447890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad333-e2d1-4da4-b2fc-867272d1a1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:58.448214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad333-e2d1-4da4-b2fc-867272d1a1b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:58.458333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ad333-e2d1-4da4-b2fc-867272d1a1b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ce4ad333-e2d1-4da4-b2fc-867272d1a1b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:58.463237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:58.464418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:14:58.465700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 602/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:14:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:58.470409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3d12e-997b-44d6-84fd-fdbb1ed3cd91 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:58.511205 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:58.568115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3d12e-997b-44d6-84fd-fdbb1ed3cd91 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:58.600741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e2fef-e867-4a7a-bca1-365f823922e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:58.601336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e2fef-e867-4a7a-bca1-365f823922e4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:58.601648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e2fef-e867-4a7a-bca1-365f823922e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:58.601923 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e2fef-e867-4a7a-bca1-365f823922e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:58.602579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e2fef-e867-4a7a-bca1-365f823922e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:58.602882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e2fef-e867-4a7a-bca1-365f823922e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:58.604770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e2fef-e867-4a7a-bca1-365f823922e4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:58.612546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 579/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:14:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:58.618323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad333-e2d1-4da4-b2fc-867272d1a1b9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:58.717833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad333-e2d1-4da4-b2fc-867272d1a1b9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:58.750708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd974cf-ce31-4037-83fe-a97d1edfa2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:58.751330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd974cf-ce31-4037-83fe-a97d1edfa2ed None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:58.751664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd974cf-ce31-4037-83fe-a97d1edfa2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:58.751938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd974cf-ce31-4037-83fe-a97d1edfa2ed None admin] PATH_INFO: `/v3/users/6cb26fc38e594933801560aabe0885a6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:58.765384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd974cf-ce31-4037-83fe-a97d1edfa2ed None admin] RBAC: Authorizing `identity:get_user(user_id=6cb26fc38e594933801560aabe0885a6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:58.766838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd974cf-ce31-4037-83fe-a97d1edfa2ed None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:58.768404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 603/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:14:58 2022] GET /identity/v3/users/6cb26fc38e594933801560aabe0885a6 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:58.774390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e2fef-e867-4a7a-bca1-365f823922e4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:58.872240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e2fef-e867-4a7a-bca1-365f823922e4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:58.873872 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:58.906020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d254d8-b291-497d-8da2-14c2fa669dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:58.906611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d254d8-b291-497d-8da2-14c2fa669dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:58.906887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d254d8-b291-497d-8da2-14c2fa669dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:58.907155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d254d8-b291-497d-8da2-14c2fa669dc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:58.972566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d254d8-b291-497d-8da2-14c2fa669dc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:58.974135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d254d8-b291-497d-8da2-14c2fa669dc7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:59.070465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 580/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:14:59.076639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd974cf-ce31-4037-83fe-a97d1edfa2ed None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:59.110629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:59.174744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd974cf-ce31-4037-83fe-a97d1edfa2ed None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:59.207738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1851540-5880-4ab0-9be5-997339fa7c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:59.208356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1851540-5880-4ab0-9be5-997339fa7c8e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:59.208633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1851540-5880-4ab0-9be5-997339fa7c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:59.208904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1851540-5880-4ab0-9be5-997339fa7c8e None admin] PATH_INFO: `/v3/projects/c9d0fdac519e4141ae06844603631029` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:59.210033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1851540-5880-4ab0-9be5-997339fa7c8e None admin] RBAC: Authorizing `identity:get_project(project_id=c9d0fdac519e4141ae06844603631029)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:59.211471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1851540-5880-4ab0-9be5-997339fa7c8e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:59.213108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 604/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:14:59 2022] GET /identity/v3/projects/c9d0fdac519e4141ae06844603631029 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:59.219364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d254d8-b291-497d-8da2-14c2fa669dc7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:59.221535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:59.317940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d254d8-b291-497d-8da2-14c2fa669dc7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:59.357388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e74787-dbfd-466c-9725-8d24eef6812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:59.358296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e74787-dbfd-466c-9725-8d24eef6812a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:59.358718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e74787-dbfd-466c-9725-8d24eef6812a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:59.359132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e74787-dbfd-466c-9725-8d24eef6812a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:59.360258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e74787-dbfd-466c-9725-8d24eef6812a None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=c9d0fdac519e4141ae06844603631029, user.id=6cb26fc38e594933801560aabe0885a6) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:59.360733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e74787-dbfd-466c-9725-8d24eef6812a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:59.363019 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e74787-dbfd-466c-9725-8d24eef6812a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:59.371437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 581/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:59 2022] GET /identity/v3/role_assignments?user.id=6cb26fc38e594933801560aabe0885a6&scope.project.id=c9d0fdac519e4141ae06844603631029&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:15:01.242577 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4e1cd-d6da-42c8-a8e4-00ffa6f620e9 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:01.243104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4e1cd-d6da-42c8-a8e4-00ffa6f620e9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:01.243104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4e1cd-d6da-42c8-a8e4-00ffa6f620e9 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:01.244981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 605/1186] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:15:01.250039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18756a37-ad9b-4736-8a34-2fd3f0d120dd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:01.250431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18756a37-ad9b-4736-8a34-2fd3f0d120dd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:01.250589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18756a37-ad9b-4736-8a34-2fd3f0d120dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:01.268996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-18756a37-ad9b-4736-8a34-2fd3f0d120dd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:15:01.270219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-18756a37-ad9b-4736-8a34-2fd3f0d120dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:15:01.329285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:15:01.336328 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 582/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:15:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:15:01.603866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4e1cd-d6da-42c8-a8e4-00ffa6f620e9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:01.702256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4e1cd-d6da-42c8-a8e4-00ffa6f620e9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:01.703739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:15:01.738693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f44ec-a409-441c-a961-86c79c00cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:01.739316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f44ec-a409-441c-a961-86c79c00cceb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:01.739588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f44ec-a409-441c-a961-86c79c00cceb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:01.739853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f44ec-a409-441c-a961-86c79c00cceb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:01.750037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f44ec-a409-441c-a961-86c79c00cceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0d1f44ec-a409-441c-a961-86c79c00cceb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:15:01.754793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:15:01.755919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:15:01.757035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 606/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:15:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:15:01.762200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18756a37-ad9b-4736-8a34-2fd3f0d120dd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:01.860154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18756a37-ad9b-4736-8a34-2fd3f0d120dd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:01.892546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b7846-9671-416e-a0cc-cfd51fd4a801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:01.893142 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b7846-9671-416e-a0cc-cfd51fd4a801 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:01.893415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b7846-9671-416e-a0cc-cfd51fd4a801 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:01.893868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b7846-9671-416e-a0cc-cfd51fd4a801 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:01.894518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b7846-9671-416e-a0cc-cfd51fd4a801 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:15:01.894816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b7846-9671-416e-a0cc-cfd51fd4a801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:15:01.896166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b7846-9671-416e-a0cc-cfd51fd4a801 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:15:01.904703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 583/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:15:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:15:01.910508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f44ec-a409-441c-a961-86c79c00cceb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:02.006467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f44ec-a409-441c-a961-86c79c00cceb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:02.038967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117019c-2e37-4e90-b655-e06c97f19a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:02.039577 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117019c-2e37-4e90-b655-e06c97f19a95 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:02.039853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117019c-2e37-4e90-b655-e06c97f19a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:02.040122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117019c-2e37-4e90-b655-e06c97f19a95 None admin] PATH_INFO: `/v3/users/6cb26fc38e594933801560aabe0885a6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:02.041416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117019c-2e37-4e90-b655-e06c97f19a95 None admin] RBAC: Authorizing `identity:get_user(user_id=6cb26fc38e594933801560aabe0885a6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:15:02.042857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117019c-2e37-4e90-b655-e06c97f19a95 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:15:02.044390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 607/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:15:01 2022] GET /identity/v3/users/6cb26fc38e594933801560aabe0885a6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:15:02.050058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b7846-9671-416e-a0cc-cfd51fd4a801 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:02.108318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:15:02.149278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b7846-9671-416e-a0cc-cfd51fd4a801 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:02.182758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f31063-21d3-47b7-a104-dea3e0969158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:02.183364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f31063-21d3-47b7-a104-dea3e0969158 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:02.183637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f31063-21d3-47b7-a104-dea3e0969158 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:02.183935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f31063-21d3-47b7-a104-dea3e0969158 None admin] PATH_INFO: `/v3/projects/c9d0fdac519e4141ae06844603631029` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:02.185019 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f31063-21d3-47b7-a104-dea3e0969158 None admin] RBAC: Authorizing `identity:get_project(project_id=c9d0fdac519e4141ae06844603631029)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:15:02.186466 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f31063-21d3-47b7-a104-dea3e0969158 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:15:02.188078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 584/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:15:02 2022] GET /identity/v3/projects/c9d0fdac519e4141ae06844603631029 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:15:02.193962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117019c-2e37-4e90-b655-e06c97f19a95 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:02.225313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:15:02.293177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117019c-2e37-4e90-b655-e06c97f19a95 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:02.326563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945be8b2-58a1-4c24-b72a-ee74900ba74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:02.330769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be8b2-58a1-4c24-b72a-ee74900ba74e None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:02.330769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be8b2-58a1-4c24-b72a-ee74900ba74e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:02.330769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be8b2-58a1-4c24-b72a-ee74900ba74e None admin] PATH_INFO: `/v3/projects/c9d0fdac519e4141ae06844603631029/users/6cb26fc38e594933801560aabe0885a6/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:02.330769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945be8b2-58a1-4c24-b72a-ee74900ba74e None admin] RBAC: Authorizing `identity:create_grant(project_id=c9d0fdac519e4141ae06844603631029, user_id=6cb26fc38e594933801560aabe0885a6, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:15:02.331135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945be8b2-58a1-4c24-b72a-ee74900ba74e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:15:02.379295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 608/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:15:02 2022] PUT /identity/v3/projects/c9d0fdac519e4141ae06844603631029/users/6cb26fc38e594933801560aabe0885a6/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:15:04.247562 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fd31a-9941-4460-b455-af865ddf84da None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:04.248095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fd31a-9941-4460-b455-af865ddf84da None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:04.248095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fd31a-9941-4460-b455-af865ddf84da None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:04.249437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 585/1193] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:15:04.254466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc48b0-46f4-4ac5-a533-dcddfc46f0f4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:04.254897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc48b0-46f4-4ac5-a533-dcddfc46f0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:04.255002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc48b0-46f4-4ac5-a533-dcddfc46f0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:04.272956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9ebc48b0-46f4-4ac5-a533-dcddfc46f0f4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:15:04.274172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9ebc48b0-46f4-4ac5-a533-dcddfc46f0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:15:04.342820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 609/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:15:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:15:04.608815 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fd31a-9941-4460-b455-af865ddf84da None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:04.709265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fd31a-9941-4460-b455-af865ddf84da None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:04.752581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:15:04.761346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955294a-8f60-49bb-8a1b-8cc0f35d24b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:04.761997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955294a-8f60-49bb-8a1b-8cc0f35d24b3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:04.762272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955294a-8f60-49bb-8a1b-8cc0f35d24b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:04.762540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955294a-8f60-49bb-8a1b-8cc0f35d24b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:04.772327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955294a-8f60-49bb-8a1b-8cc0f35d24b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-4955294a-8f60-49bb-8a1b-8cc0f35d24b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:15:04.777577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:15:04.778666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:15:04.779864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 586/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:15:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:15:04.784735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc48b0-46f4-4ac5-a533-dcddfc46f0f4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:04.807487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:15:04.884739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc48b0-46f4-4ac5-a533-dcddfc46f0f4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:04.918797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbfa9e-3aec-4f9e-8758-04915897a851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:04.919394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbfa9e-3aec-4f9e-8758-04915897a851 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:04.919676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbfa9e-3aec-4f9e-8758-04915897a851 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:04.919950 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbfa9e-3aec-4f9e-8758-04915897a851 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:04.920610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbfa9e-3aec-4f9e-8758-04915897a851 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:15:04.920915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbfa9e-3aec-4f9e-8758-04915897a851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:15:04.922300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbfa9e-3aec-4f9e-8758-04915897a851 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:15:04.930170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 610/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:15:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:15:04.936000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955294a-8f60-49bb-8a1b-8cc0f35d24b3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:05.035290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955294a-8f60-49bb-8a1b-8cc0f35d24b3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:05.070929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaad18-abe0-4c11-ae18-f0ef261213cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:05.071767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaad18-abe0-4c11-ae18-f0ef261213cf None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:05.072185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaad18-abe0-4c11-ae18-f0ef261213cf None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:05.072458 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaad18-abe0-4c11-ae18-f0ef261213cf None admin] PATH_INFO: `/v3/users/6cb26fc38e594933801560aabe0885a6` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:05.073589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaad18-abe0-4c11-ae18-f0ef261213cf None admin] RBAC: Authorizing `identity:get_user(user_id=6cb26fc38e594933801560aabe0885a6)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:15:05.075016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaad18-abe0-4c11-ae18-f0ef261213cf None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:15:05.076577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 587/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 1 05:15:04 2022] GET /identity/v3/users/6cb26fc38e594933801560aabe0885a6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:15:05.082523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbfa9e-3aec-4f9e-8758-04915897a851 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:05.163057 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:15:05.180345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbfa9e-3aec-4f9e-8758-04915897a851 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:05.214292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d36761-e859-4317-a4f8-783d897d1b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:05.214895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d36761-e859-4317-a4f8-783d897d1b19 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:05.215177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d36761-e859-4317-a4f8-783d897d1b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:05.215453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d36761-e859-4317-a4f8-783d897d1b19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:05.280805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d36761-e859-4317-a4f8-783d897d1b19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:15:05.282264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d36761-e859-4317-a4f8-783d897d1b19 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:15:05.359866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:15:05.377817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 611/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:15:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:15:05.384325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaad18-abe0-4c11-ae18-f0ef261213cf None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:05.441057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:15:05.486110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaad18-abe0-4c11-ae18-f0ef261213cf None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:05.518590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299386b8-90c0-41e5-8968-390808d45df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:05.519196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299386b8-90c0-41e5-8968-390808d45df7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:05.519469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299386b8-90c0-41e5-8968-390808d45df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:05.519734 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299386b8-90c0-41e5-8968-390808d45df7 None admin] PATH_INFO: `/v3/projects/c9d0fdac519e4141ae06844603631029` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:05.520819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299386b8-90c0-41e5-8968-390808d45df7 None admin] RBAC: Authorizing `identity:get_project(project_id=c9d0fdac519e4141ae06844603631029)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:15:05.522272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299386b8-90c0-41e5-8968-390808d45df7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:15:05.523876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 588/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 1 05:15:05 2022] GET /identity/v3/projects/c9d0fdac519e4141ae06844603631029 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:15:05.530664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d36761-e859-4317-a4f8-783d897d1b19 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:15:05.626935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d36761-e859-4317-a4f8-783d897d1b19 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:15:05.659803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487ba1e-ac97-40dd-9d86-4b28c537e62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:15:05.660391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487ba1e-ac97-40dd-9d86-4b28c537e62e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:15:05.660666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487ba1e-ac97-40dd-9d86-4b28c537e62e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:15:05.660937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487ba1e-ac97-40dd-9d86-4b28c537e62e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:15:05.661685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487ba1e-ac97-40dd-9d86-4b28c537e62e None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=c9d0fdac519e4141ae06844603631029, user.id=6cb26fc38e594933801560aabe0885a6) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:15:05.661993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487ba1e-ac97-40dd-9d86-4b28c537e62e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:15:05.663405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487ba1e-ac97-40dd-9d86-4b28c537e62e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:15:05.669940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 612/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:15:05 2022] GET /identity/v3/role_assignments?user.id=6cb26fc38e594933801560aabe0885a6&scope.project.id=c9d0fdac519e4141ae06844603631029&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:15:15.514533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=39 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:15:15.667333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=77 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:16.049656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912597e-0d86-4a6d-9107-b1cbce5fd3d4 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:16.050214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912597e-0d86-4a6d-9107-b1cbce5fd3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:16.050214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912597e-0d86-4a6d-9107-b1cbce5fd3d4 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:16.051406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 589/1201] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:16.057060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf27542-3137-426e-8889-b9639c5ef0f6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:16.057539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf27542-3137-426e-8889-b9639c5ef0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:16.057637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf27542-3137-426e-8889-b9639c5ef0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:16.076494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fdf27542-3137-426e-8889-b9639c5ef0f6 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:16.077764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fdf27542-3137-426e-8889-b9639c5ef0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:16.146854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 613/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:16:16.424952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912597e-0d86-4a6d-9107-b1cbce5fd3d4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:16.537403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912597e-0d86-4a6d-9107-b1cbce5fd3d4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:16.572282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14377dcc-d46f-494d-9de8-18c2e48baaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:16.572904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14377dcc-d46f-494d-9de8-18c2e48baaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:16.573178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14377dcc-d46f-494d-9de8-18c2e48baaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:16.573445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14377dcc-d46f-494d-9de8-18c2e48baaa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:16.583160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14377dcc-d46f-494d-9de8-18c2e48baaa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:16.584605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14377dcc-d46f-494d-9de8-18c2e48baaa9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-14377dcc-d46f-494d-9de8-18c2e48baaa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:16.599468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:16.604203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:16.605421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:16.605421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:16.605421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:16.605421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 590/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:16.607224 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf27542-3137-426e-8889-b9639c5ef0f6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:16.719437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf27542-3137-426e-8889-b9639c5ef0f6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:16.748858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:16.754897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902486c-f0b7-4542-86c6-ff5d66ecd8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:16.755508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902486c-f0b7-4542-86c6-ff5d66ecd8bc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:16.755787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902486c-f0b7-4542-86c6-ff5d66ecd8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:16.756060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902486c-f0b7-4542-86c6-ff5d66ecd8bc None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:16.756848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902486c-f0b7-4542-86c6-ff5d66ecd8bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:16.757159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902486c-f0b7-4542-86c6-ff5d66ecd8bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:16.758554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902486c-f0b7-4542-86c6-ff5d66ecd8bc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:16.769854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 614/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:16.776624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14377dcc-d46f-494d-9de8-18c2e48baaa9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:16.832294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:16.880551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14377dcc-d46f-494d-9de8-18c2e48baaa9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:16.914987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25794ac-9b6b-4809-a2a7-6580bd1b6d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:16.915581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25794ac-9b6b-4809-a2a7-6580bd1b6d30 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:16.915855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25794ac-9b6b-4809-a2a7-6580bd1b6d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:16.916120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25794ac-9b6b-4809-a2a7-6580bd1b6d30 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:16.916759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25794ac-9b6b-4809-a2a7-6580bd1b6d30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:16.918203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25794ac-9b6b-4809-a2a7-6580bd1b6d30 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:16.981601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 591/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 1 05:16:16 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:16:18.922079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa701e-aa08-448d-94d3-f98a16aaa92c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:18.922658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa701e-aa08-448d-94d3-f98a16aaa92c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:18.922658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa701e-aa08-448d-94d3-f98a16aaa92c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:18.923963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 615/1206] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:18.929970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53987343-c083-4b81-ba07-82b310af8e34 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:18.932289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53987343-c083-4b81-ba07-82b310af8e34 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:18.932289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53987343-c083-4b81-ba07-82b310af8e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:18.949969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-53987343-c083-4b81-ba07-82b310af8e34 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:18.951165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-53987343-c083-4b81-ba07-82b310af8e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:19.017999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 592/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:16:19.296372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa701e-aa08-448d-94d3-f98a16aaa92c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:19.399928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa701e-aa08-448d-94d3-f98a16aaa92c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:19.432151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513974d-dd9d-4199-921a-14e359a40fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:19.432787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513974d-dd9d-4199-921a-14e359a40fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:19.433057 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513974d-dd9d-4199-921a-14e359a40fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:19.433328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513974d-dd9d-4199-921a-14e359a40fb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:19.444584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513974d-dd9d-4199-921a-14e359a40fb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a513974d-dd9d-4199-921a-14e359a40fb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:16:19.449277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:19.450429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:19.451680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 616/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:16:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:19.456516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53987343-c083-4b81-ba07-82b310af8e34 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:19.497833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:19.557260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53987343-c083-4b81-ba07-82b310af8e34 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:19.591304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce3273-3c7c-4c2b-8aa3-74977f6bec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:19.592108 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce3273-3c7c-4c2b-8aa3-74977f6bec5d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:19.592373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce3273-3c7c-4c2b-8aa3-74977f6bec5d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:19.592637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce3273-3c7c-4c2b-8aa3-74977f6bec5d None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:19.593282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce3273-3c7c-4c2b-8aa3-74977f6bec5d None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:19.593631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce3273-3c7c-4c2b-8aa3-74977f6bec5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:19.594994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce3273-3c7c-4c2b-8aa3-74977f6bec5d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:19.603486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 593/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:16:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:19.609311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513974d-dd9d-4199-921a-14e359a40fb9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:19.659124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:19.709758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513974d-dd9d-4199-921a-14e359a40fb9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:19.745990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191c49-3fb7-4df6-b944-f2348444abad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:19.750504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191c49-3fb7-4df6-b944-f2348444abad None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:19.750504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191c49-3fb7-4df6-b944-f2348444abad None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:19.750504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191c49-3fb7-4df6-b944-f2348444abad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:19.758343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48191c49-3fb7-4df6-b944-f2348444abad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:19.759821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48191c49-3fb7-4df6-b944-f2348444abad None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-48191c49-3fb7-4df6-b944-f2348444abad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:19.773833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:19.780367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:19.781531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:19.781531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:19.781531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:19.781531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 617/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:19.781898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce3273-3c7c-4c2b-8aa3-74977f6bec5d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:19.880342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:19.884098 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce3273-3c7c-4c2b-8aa3-74977f6bec5d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:19.916233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a7281-cc91-4482-92b8-e0b9eed7bfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:19.916821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a7281-cc91-4482-92b8-e0b9eed7bfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:19.917093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a7281-cc91-4482-92b8-e0b9eed7bfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:19.917356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a7281-cc91-4482-92b8-e0b9eed7bfe9 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:19.918008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a7281-cc91-4482-92b8-e0b9eed7bfe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:19.918303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a7281-cc91-4482-92b8-e0b9eed7bfe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:19.919651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a7281-cc91-4482-92b8-e0b9eed7bfe9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:19.931086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 594/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:19.936883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191c49-3fb7-4df6-b944-f2348444abad None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:20.041188 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191c49-3fb7-4df6-b944-f2348444abad None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:20.042794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:20.074366 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f359253-7bf1-4db9-ba38-b70fc510529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:20.074982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f359253-7bf1-4db9-ba38-b70fc510529c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:20.075257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f359253-7bf1-4db9-ba38-b70fc510529c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:20.075525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f359253-7bf1-4db9-ba38-b70fc510529c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:20.082885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f359253-7bf1-4db9-ba38-b70fc510529c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:20.084302 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f359253-7bf1-4db9-ba38-b70fc510529c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1f359253-7bf1-4db9-ba38-b70fc510529c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:16:20.095352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:20.096498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:20.097608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 618/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:20.104864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a7281-cc91-4482-92b8-e0b9eed7bfe9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:20.202511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a7281-cc91-4482-92b8-e0b9eed7bfe9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:20.236044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1641ef-e516-402e-bffe-94107bf21a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:20.236673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1641ef-e516-402e-bffe-94107bf21a31 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:20.236945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1641ef-e516-402e-bffe-94107bf21a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:20.237214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1641ef-e516-402e-bffe-94107bf21a31 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:20.237960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1641ef-e516-402e-bffe-94107bf21a31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:20.238264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1641ef-e516-402e-bffe-94107bf21a31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:20.239686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1641ef-e516-402e-bffe-94107bf21a31 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:20.257747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 595/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 1 05:16:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:20.263797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f359253-7bf1-4db9-ba38-b70fc510529c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:20.362221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f359253-7bf1-4db9-ba38-b70fc510529c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:20.395265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ee963-2dde-4e4d-abcc-f8f2c2d19ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:20.395867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ee963-2dde-4e4d-abcc-f8f2c2d19ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:20.396144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ee963-2dde-4e4d-abcc-f8f2c2d19ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:20.396411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ee963-2dde-4e4d-abcc-f8f2c2d19ab6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:20.431485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:20.469225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ee963-2dde-4e4d-abcc-f8f2c2d19ab6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:20.470622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ee963-2dde-4e4d-abcc-f8f2c2d19ab6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:20.571115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 619/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:16:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:16:20.577270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1641ef-e516-402e-bffe-94107bf21a31 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:20.596688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:20.680890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1641ef-e516-402e-bffe-94107bf21a31 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:20.717603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d282db7-f7c3-4070-9dd8-08c3547a32a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:20.718662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d282db7-f7c3-4070-9dd8-08c3547a32a4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:20.719159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d282db7-f7c3-4070-9dd8-08c3547a32a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:20.719685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d282db7-f7c3-4070-9dd8-08c3547a32a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:20.734889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d282db7-f7c3-4070-9dd8-08c3547a32a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:20.736373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d282db7-f7c3-4070-9dd8-08c3547a32a4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-3d282db7-f7c3-4070-9dd8-08c3547a32a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:20.750689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:20.751825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:20.752926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:20.752926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:20.752926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:20.752926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 596/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:16:20.758202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ee963-2dde-4e4d-abcc-f8f2c2d19ab6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:20.817860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:20.857973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ee963-2dde-4e4d-abcc-f8f2c2d19ab6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:20.891461 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e736d-9866-4585-ae8d-350060ced628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:20.892068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e736d-9866-4585-ae8d-350060ced628 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:20.892346 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e736d-9866-4585-ae8d-350060ced628 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:20.892617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e736d-9866-4585-ae8d-350060ced628 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:20.893246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e736d-9866-4585-ae8d-350060ced628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:20.893619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e736d-9866-4585-ae8d-350060ced628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:20.894978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e736d-9866-4585-ae8d-350060ced628 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:20.907068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 620/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:20.912770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d282db7-f7c3-4070-9dd8-08c3547a32a4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:20.983486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:21.014943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d282db7-f7c3-4070-9dd8-08c3547a32a4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:21.049027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e5aa4-5817-4203-bbdc-14692ddf9af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:21.049710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e5aa4-5817-4203-bbdc-14692ddf9af5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:21.049989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e5aa4-5817-4203-bbdc-14692ddf9af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:21.050280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e5aa4-5817-4203-bbdc-14692ddf9af5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:21.057667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e5aa4-5817-4203-bbdc-14692ddf9af5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:21.059174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e5aa4-5817-4203-bbdc-14692ddf9af5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-be3e5aa4-5817-4203-bbdc-14692ddf9af5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:16:21.072831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:21.074101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:21.075145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:16:21.075145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:16:21.075145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:16:21.075145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:16:21.075145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:16:21.075145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:16:21.075145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:21.075145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:21.075145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 597/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:16:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:21.080109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e736d-9866-4585-ae8d-350060ced628 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:21.188684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e736d-9866-4585-ae8d-350060ced628 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:21.212446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:21.223755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f245d9b-b149-448f-a13f-ca3bcd1384ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:21.224354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f245d9b-b149-448f-a13f-ca3bcd1384ba None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:21.224626 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f245d9b-b149-448f-a13f-ca3bcd1384ba None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:21.224891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f245d9b-b149-448f-a13f-ca3bcd1384ba None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:21.226253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f245d9b-b149-448f-a13f-ca3bcd1384ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:21.226554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f245d9b-b149-448f-a13f-ca3bcd1384ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:21.227966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f245d9b-b149-448f-a13f-ca3bcd1384ba None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:21.239677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 621/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:16:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:21.245835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e5aa4-5817-4203-bbdc-14692ddf9af5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:21.346804 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e5aa4-5817-4203-bbdc-14692ddf9af5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:21.375195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:21.380854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa39035-0607-4ec4-8909-20cb23265fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:21.381474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa39035-0607-4ec4-8909-20cb23265fcd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:21.381761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa39035-0607-4ec4-8909-20cb23265fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:21.382033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa39035-0607-4ec4-8909-20cb23265fcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:21.382748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa39035-0607-4ec4-8909-20cb23265fcd None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=c11c7fb612ab40d88f526b8b38dbfb23) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:21.383053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa39035-0607-4ec4-8909-20cb23265fcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:21.384496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa39035-0607-4ec4-8909-20cb23265fcd None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:21.389742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 598/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:16:21 2022] GET /identity/v3/role_assignments?user.id=c11c7fb612ab40d88f526b8b38dbfb23&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:23.273858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b598a-d9cd-40e2-998c-3f4dc033984d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:23.274421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b598a-d9cd-40e2-998c-3f4dc033984d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:23.274421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b598a-d9cd-40e2-998c-3f4dc033984d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:23.275735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 622/1220] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:23.280744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94882c-dd52-48ae-bf2b-68d5d3d7786f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:23.281126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94882c-dd52-48ae-bf2b-68d5d3d7786f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:23.281274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94882c-dd52-48ae-bf2b-68d5d3d7786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:23.299430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2d94882c-dd52-48ae-bf2b-68d5d3d7786f None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:23.300606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2d94882c-dd52-48ae-bf2b-68d5d3d7786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:23.366837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 599/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:16:23.639651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b598a-d9cd-40e2-998c-3f4dc033984d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:23.744720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b598a-d9cd-40e2-998c-3f4dc033984d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:23.779851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d447b40-bc0a-4385-a54e-7181ef4d90ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:23.780482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447b40-bc0a-4385-a54e-7181ef4d90ef None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:23.780762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447b40-bc0a-4385-a54e-7181ef4d90ef None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:23.781033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447b40-bc0a-4385-a54e-7181ef4d90ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:23.791084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d447b40-bc0a-4385-a54e-7181ef4d90ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6d447b40-bc0a-4385-a54e-7181ef4d90ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:16:23.795908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:23.797002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:23.798352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 623/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:16:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:23.803266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94882c-dd52-48ae-bf2b-68d5d3d7786f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:23.902647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94882c-dd52-48ae-bf2b-68d5d3d7786f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:23.925960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:23.936657 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb4fd1-06ed-45e4-8fad-d0e301e239f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:23.937258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb4fd1-06ed-45e4-8fad-d0e301e239f1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:23.937560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb4fd1-06ed-45e4-8fad-d0e301e239f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:23.937832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb4fd1-06ed-45e4-8fad-d0e301e239f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:23.938483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb4fd1-06ed-45e4-8fad-d0e301e239f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:23.938781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb4fd1-06ed-45e4-8fad-d0e301e239f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:23.940201 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb4fd1-06ed-45e4-8fad-d0e301e239f1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:23.949152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 600/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:16:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:23.955011 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d447b40-bc0a-4385-a54e-7181ef4d90ef None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:23.999439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:24.055855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d447b40-bc0a-4385-a54e-7181ef4d90ef None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:24.090864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033c0a5-5bb2-4cf7-a55f-0066b905a2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:24.091509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033c0a5-5bb2-4cf7-a55f-0066b905a2fa None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:24.091772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033c0a5-5bb2-4cf7-a55f-0066b905a2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:24.092045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033c0a5-5bb2-4cf7-a55f-0066b905a2fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:24.102046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033c0a5-5bb2-4cf7-a55f-0066b905a2fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:24.103524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033c0a5-5bb2-4cf7-a55f-0066b905a2fa None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8033c0a5-5bb2-4cf7-a55f-0066b905a2fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:24.118457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:24.119833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:24.121037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:24.121037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:24.121037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:24.121037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 624/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:24.125411 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb4fd1-06ed-45e4-8fad-d0e301e239f1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:24.226165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb4fd1-06ed-45e4-8fad-d0e301e239f1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:24.267232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bdb9a-af2e-4194-bbe7-6c501d5c9043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:24.267831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bdb9a-af2e-4194-bbe7-6c501d5c9043 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:24.268107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bdb9a-af2e-4194-bbe7-6c501d5c9043 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:24.268380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bdb9a-af2e-4194-bbe7-6c501d5c9043 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:24.269011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bdb9a-af2e-4194-bbe7-6c501d5c9043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:24.269310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bdb9a-af2e-4194-bbe7-6c501d5c9043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:24.270712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bdb9a-af2e-4194-bbe7-6c501d5c9043 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:24.282546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 601/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:24.288725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033c0a5-5bb2-4cf7-a55f-0066b905a2fa None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:24.383615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:24.390523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033c0a5-5bb2-4cf7-a55f-0066b905a2fa None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:24.425052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c66e4-6fc3-4a32-b134-f18ab6ec39c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:24.425712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c66e4-6fc3-4a32-b134-f18ab6ec39c2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:24.425992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c66e4-6fc3-4a32-b134-f18ab6ec39c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:24.426264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c66e4-6fc3-4a32-b134-f18ab6ec39c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:24.433992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c66e4-6fc3-4a32-b134-f18ab6ec39c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:24.435414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c66e4-6fc3-4a32-b134-f18ab6ec39c2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-593c66e4-6fc3-4a32-b134-f18ab6ec39c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:16:24.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:24.448114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:24.449201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 625/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:24.454175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bdb9a-af2e-4194-bbe7-6c501d5c9043 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:24.503916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:24.561323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bdb9a-af2e-4194-bbe7-6c501d5c9043 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:24.596338 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582f60d-243b-4b00-b125-44b70bd2b45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:24.599095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582f60d-243b-4b00-b125-44b70bd2b45a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:24.599095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582f60d-243b-4b00-b125-44b70bd2b45a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:24.599095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582f60d-243b-4b00-b125-44b70bd2b45a None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:24.599095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582f60d-243b-4b00-b125-44b70bd2b45a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:24.599095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582f60d-243b-4b00-b125-44b70bd2b45a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:24.600144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582f60d-243b-4b00-b125-44b70bd2b45a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:24.618378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 602/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 1 05:16:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:24.624136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c66e4-6fc3-4a32-b134-f18ab6ec39c2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:24.731221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c66e4-6fc3-4a32-b134-f18ab6ec39c2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:24.765145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2302544-b7a9-4f08-8499-c88c35595391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:24.765747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2302544-b7a9-4f08-8499-c88c35595391 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:24.766020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2302544-b7a9-4f08-8499-c88c35595391 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:24.766293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2302544-b7a9-4f08-8499-c88c35595391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:24.777353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2302544-b7a9-4f08-8499-c88c35595391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:24.778963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2302544-b7a9-4f08-8499-c88c35595391 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e2302544-b7a9-4f08-8499-c88c35595391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:24.792640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:24.793787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:24.794741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:24.794741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:24.794741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:24.794741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 626/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:24.799656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582f60d-243b-4b00-b125-44b70bd2b45a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:24.900870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582f60d-243b-4b00-b125-44b70bd2b45a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:24.903195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:24.934984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eda962-49fc-48e4-a669-2f6d7c728038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:24.935574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eda962-49fc-48e4-a669-2f6d7c728038 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:24.935847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eda962-49fc-48e4-a669-2f6d7c728038 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:24.936113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eda962-49fc-48e4-a669-2f6d7c728038 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:24.936750 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eda962-49fc-48e4-a669-2f6d7c728038 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:24.937050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eda962-49fc-48e4-a669-2f6d7c728038 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:24.938520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eda962-49fc-48e4-a669-2f6d7c728038 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:24.950776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 603/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:24.956541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2302544-b7a9-4f08-8499-c88c35595391 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:24.966335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:25.057008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2302544-b7a9-4f08-8499-c88c35595391 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:25.092104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea80a77-08ba-4037-9ed4-174516369391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:25.092739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea80a77-08ba-4037-9ed4-174516369391 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:25.093018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea80a77-08ba-4037-9ed4-174516369391 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:25.093305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea80a77-08ba-4037-9ed4-174516369391 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:25.099881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea80a77-08ba-4037-9ed4-174516369391 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:25.101549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea80a77-08ba-4037-9ed4-174516369391 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-fea80a77-08ba-4037-9ed4-174516369391 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:16:25.112556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:25.113660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:25.114589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:16:25.114589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:16:25.114589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:16:25.114589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:16:25.114589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:16:25.114589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:16:25.114589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:25.114589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:25.114589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 627/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:25.120200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eda962-49fc-48e4-a669-2f6d7c728038 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:25.220840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eda962-49fc-48e4-a669-2f6d7c728038 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:25.254124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10be41d-cbe6-49ad-9846-444ac3beaaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:25.254719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10be41d-cbe6-49ad-9846-444ac3beaaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:25.254996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10be41d-cbe6-49ad-9846-444ac3beaaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:25.255265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10be41d-cbe6-49ad-9846-444ac3beaaa5 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:25.256474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10be41d-cbe6-49ad-9846-444ac3beaaa5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:25.256848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10be41d-cbe6-49ad-9846-444ac3beaaa5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:25.258294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10be41d-cbe6-49ad-9846-444ac3beaaa5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:25.270525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 604/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:16:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:25.277376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea80a77-08ba-4037-9ed4-174516369391 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:25.342637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:25.375954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea80a77-08ba-4037-9ed4-174516369391 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:25.410216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f783c-d90d-470b-8ba9-87f346bf895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:25.410871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f783c-d90d-470b-8ba9-87f346bf895b None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:25.411152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f783c-d90d-470b-8ba9-87f346bf895b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:25.411426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f783c-d90d-470b-8ba9-87f346bf895b None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/c11c7fb612ab40d88f526b8b38dbfb23/roles/c6689cf082df4be9b69f2208c097f2c3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:25.425655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f783c-d90d-470b-8ba9-87f346bf895b None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=c11c7fb612ab40d88f526b8b38dbfb23, role_id=c6689cf082df4be9b69f2208c097f2c3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:25.427238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f783c-d90d-470b-8ba9-87f346bf895b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:25.512128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 628/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:16:25 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/c11c7fb612ab40d88f526b8b38dbfb23/roles/c6689cf082df4be9b69f2208c097f2c3 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:16:27.407392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20dfd9-c81a-4630-9def-038512ec1ae6 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:27.407945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20dfd9-c81a-4630-9def-038512ec1ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:27.407945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20dfd9-c81a-4630-9def-038512ec1ae6 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:27.409290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 605/1233] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:27.414738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78e0bc-c1c2-4d3c-9cc5-90722214e93e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:27.415145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78e0bc-c1c2-4d3c-9cc5-90722214e93e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:27.415292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78e0bc-c1c2-4d3c-9cc5-90722214e93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:27.433876 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3e78e0bc-c1c2-4d3c-9cc5-90722214e93e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:27.435077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3e78e0bc-c1c2-4d3c-9cc5-90722214e93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:27.502861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 629/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:16:27.801566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20dfd9-c81a-4630-9def-038512ec1ae6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:27.827464 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:27.903555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20dfd9-c81a-4630-9def-038512ec1ae6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:27.937880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d0c61-f903-485e-be33-fe4963ed69f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:27.938498 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d0c61-f903-485e-be33-fe4963ed69f4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:27.938845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d0c61-f903-485e-be33-fe4963ed69f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:27.939114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d0c61-f903-485e-be33-fe4963ed69f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:27.949693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d0c61-f903-485e-be33-fe4963ed69f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e48d0c61-f903-485e-be33-fe4963ed69f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:16:27.954517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:27.957513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:27.958402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 606/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 1 05:16:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:27.962867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78e0bc-c1c2-4d3c-9cc5-90722214e93e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:28.008683 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:28.061245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78e0bc-c1c2-4d3c-9cc5-90722214e93e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:28.095958 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bab4a-7b98-40bb-afc4-b6764759a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:28.096561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bab4a-7b98-40bb-afc4-b6764759a0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:28.096835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bab4a-7b98-40bb-afc4-b6764759a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:28.097101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bab4a-7b98-40bb-afc4-b6764759a0f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:28.097864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bab4a-7b98-40bb-afc4-b6764759a0f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:28.098161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bab4a-7b98-40bb-afc4-b6764759a0f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:28.100186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bab4a-7b98-40bb-afc4-b6764759a0f4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:28.108035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 630/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 1 05:16:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:28.113937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d0c61-f903-485e-be33-fe4963ed69f4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:28.198524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:28.214424 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d0c61-f903-485e-be33-fe4963ed69f4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:28.248909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574f416-64dd-4d3f-aba5-f10544a55021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:28.249564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574f416-64dd-4d3f-aba5-f10544a55021 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:28.249845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574f416-64dd-4d3f-aba5-f10544a55021 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:28.250116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574f416-64dd-4d3f-aba5-f10544a55021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:28.260605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574f416-64dd-4d3f-aba5-f10544a55021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:28.262030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574f416-64dd-4d3f-aba5-f10544a55021 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-c574f416-64dd-4d3f-aba5-f10544a55021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:28.276802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:28.277974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:28.279072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:28.279072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:28.279072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:28.279072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 607/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:28.283995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bab4a-7b98-40bb-afc4-b6764759a0f4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:28.387015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bab4a-7b98-40bb-afc4-b6764759a0f4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:28.395595 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:28.420128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d0e9e-367f-483f-a27a-5df9609234eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:28.420746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d0e9e-367f-483f-a27a-5df9609234eb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:28.420985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d0e9e-367f-483f-a27a-5df9609234eb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:28.421251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d0e9e-367f-483f-a27a-5df9609234eb None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:28.421910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d0e9e-367f-483f-a27a-5df9609234eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:28.422210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d0e9e-367f-483f-a27a-5df9609234eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:28.423550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d0e9e-367f-483f-a27a-5df9609234eb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:28.435659 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 631/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:28.441949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574f416-64dd-4d3f-aba5-f10544a55021 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:28.543699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574f416-64dd-4d3f-aba5-f10544a55021 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:28.577666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd79d6-4735-4d3c-a00a-33274a19dcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:28.578282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd79d6-4735-4d3c-a00a-33274a19dcf3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:28.578553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd79d6-4735-4d3c-a00a-33274a19dcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:28.578818 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd79d6-4735-4d3c-a00a-33274a19dcf3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:28.581544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:28.586724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd79d6-4735-4d3c-a00a-33274a19dcf3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:28.588146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd79d6-4735-4d3c-a00a-33274a19dcf3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-51dd79d6-4735-4d3c-a00a-33274a19dcf3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:16:28.599004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:28.600746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:28.602254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 608/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:28.606794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d0e9e-367f-483f-a27a-5df9609234eb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:28.707142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d0e9e-367f-483f-a27a-5df9609234eb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:28.742372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db7cb8-bc40-4851-8998-07062ca461e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:28.742976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db7cb8-bc40-4851-8998-07062ca461e3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:28.743247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db7cb8-bc40-4851-8998-07062ca461e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:28.743513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db7cb8-bc40-4851-8998-07062ca461e3 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:28.744229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db7cb8-bc40-4851-8998-07062ca461e3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:28.744530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db7cb8-bc40-4851-8998-07062ca461e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:28.745980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db7cb8-bc40-4851-8998-07062ca461e3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:28.766092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 632/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 1 05:16:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:28.772504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd79d6-4735-4d3c-a00a-33274a19dcf3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:28.876633 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd79d6-4735-4d3c-a00a-33274a19dcf3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:28.909958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17856e-e153-4ece-a764-1bdf5170f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:28.910554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17856e-e153-4ece-a764-1bdf5170f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:28.910831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17856e-e153-4ece-a764-1bdf5170f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:28.911103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17856e-e153-4ece-a764-1bdf5170f7c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:28.972404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:28.978960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17856e-e153-4ece-a764-1bdf5170f7c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:28.980348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17856e-e153-4ece-a764-1bdf5170f7c5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:29.079051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 609/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:16:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:16:29.085240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db7cb8-bc40-4851-8998-07062ca461e3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:29.112982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:29.183900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db7cb8-bc40-4851-8998-07062ca461e3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:29.217050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984b9e9-0c19-4625-b330-0c2295316363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:29.217694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b9e9-0c19-4625-b330-0c2295316363 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:29.217977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b9e9-0c19-4625-b330-0c2295316363 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:29.218244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b9e9-0c19-4625-b330-0c2295316363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:29.228540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984b9e9-0c19-4625-b330-0c2295316363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:29.230000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984b9e9-0c19-4625-b330-0c2295316363 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4984b9e9-0c19-4625-b330-0c2295316363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:29.245405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:29.246546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:29.247852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:29.247852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:29.247852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:29.247852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 633/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:29.252560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17856e-e153-4ece-a764-1bdf5170f7c5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:29.347743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:29.354392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17856e-e153-4ece-a764-1bdf5170f7c5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:29.389622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344ffed-2b43-4f19-b4b8-c034854fe793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:29.390227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344ffed-2b43-4f19-b4b8-c034854fe793 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:29.390506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344ffed-2b43-4f19-b4b8-c034854fe793 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:29.390779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344ffed-2b43-4f19-b4b8-c034854fe793 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:29.391414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344ffed-2b43-4f19-b4b8-c034854fe793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:29.391714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344ffed-2b43-4f19-b4b8-c034854fe793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:29.393087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344ffed-2b43-4f19-b4b8-c034854fe793 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:29.404882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 610/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:29.410883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984b9e9-0c19-4625-b330-0c2295316363 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:29.490110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:29.510859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984b9e9-0c19-4625-b330-0c2295316363 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:29.543862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9b12e-8c91-48f2-a6bd-a1f2eb8582cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:29.544476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b12e-8c91-48f2-a6bd-a1f2eb8582cd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:29.544866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b12e-8c91-48f2-a6bd-a1f2eb8582cd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:29.545139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b12e-8c91-48f2-a6bd-a1f2eb8582cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:29.551487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9b12e-8c91-48f2-a6bd-a1f2eb8582cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:29.553222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9b12e-8c91-48f2-a6bd-a1f2eb8582cd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1ab9b12e-8c91-48f2-a6bd-a1f2eb8582cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:16:29.564909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:29.566051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:29.567352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:16:29.567352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:16:29.567352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:16:29.567352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:16:29.567352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:16:29.567352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:16:29.567352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:29.567352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:29.567352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 634/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:29.572431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344ffed-2b43-4f19-b4b8-c034854fe793 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:29.675734 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344ffed-2b43-4f19-b4b8-c034854fe793 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:29.710044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24de48-04bf-45e0-b818-dd1c4a2ff97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:29.711591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24de48-04bf-45e0-b818-dd1c4a2ff97f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:29.711591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24de48-04bf-45e0-b818-dd1c4a2ff97f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:29.711591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24de48-04bf-45e0-b818-dd1c4a2ff97f None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:29.711862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24de48-04bf-45e0-b818-dd1c4a2ff97f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:29.712114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24de48-04bf-45e0-b818-dd1c4a2ff97f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:29.713564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24de48-04bf-45e0-b818-dd1c4a2ff97f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:29.727007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 611/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:29.733205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9b12e-8c91-48f2-a6bd-a1f2eb8582cd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:29.835146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9b12e-8c91-48f2-a6bd-a1f2eb8582cd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:29.867954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9afe91-56b7-4514-a67e-b00dcd06cc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:29.868557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9afe91-56b7-4514-a67e-b00dcd06cc93 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:29.868834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9afe91-56b7-4514-a67e-b00dcd06cc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:29.869104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9afe91-56b7-4514-a67e-b00dcd06cc93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:29.869843 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9afe91-56b7-4514-a67e-b00dcd06cc93 None admin] RBAC: Adding query filter params (role.id=c6689cf082df4be9b69f2208c097f2c3, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=c11c7fb612ab40d88f526b8b38dbfb23) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:29.870151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9afe91-56b7-4514-a67e-b00dcd06cc93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:29.871624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9afe91-56b7-4514-a67e-b00dcd06cc93 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:29.877255 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 635/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:16:29 2022] GET /identity/v3/role_assignments?user.id=c11c7fb612ab40d88f526b8b38dbfb23&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=c6689cf082df4be9b69f2208c097f2c3 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:31.841593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8e99a-e045-4ed4-9988-0248fcaab464 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:31.842147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8e99a-e045-4ed4-9988-0248fcaab464 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:31.842147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8e99a-e045-4ed4-9988-0248fcaab464 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:31.843465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 612/1247] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:31.848584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda8266-a7a8-47f1-a9fa-389b0a536770 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:31.849030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda8266-a7a8-47f1-a9fa-389b0a536770 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:31.849742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda8266-a7a8-47f1-a9fa-389b0a536770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:31.855952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:31.868147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2cda8266-a7a8-47f1-a9fa-389b0a536770 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:31.869314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2cda8266-a7a8-47f1-a9fa-389b0a536770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:31.941642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 636/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:16:32.210081 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8e99a-e045-4ed4-9988-0248fcaab464 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:32.232180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:32.325667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8e99a-e045-4ed4-9988-0248fcaab464 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:32.360822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbbd5a-5bf8-4377-adf7-b76050ed11e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:32.361496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbbd5a-5bf8-4377-adf7-b76050ed11e5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:32.361778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbbd5a-5bf8-4377-adf7-b76050ed11e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:32.362051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbbd5a-5bf8-4377-adf7-b76050ed11e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:32.372561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbbd5a-5bf8-4377-adf7-b76050ed11e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-a6cbbd5a-5bf8-4377-adf7-b76050ed11e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:16:32.377269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:32.378656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:32.379729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 613/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:32.384611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda8266-a7a8-47f1-a9fa-389b0a536770 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:32.486848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda8266-a7a8-47f1-a9fa-389b0a536770 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:32.515473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:32.520155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e21888-fc56-4f77-b5f3-98db99069a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:32.520751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e21888-fc56-4f77-b5f3-98db99069a36 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:32.521616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e21888-fc56-4f77-b5f3-98db99069a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:32.521917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e21888-fc56-4f77-b5f3-98db99069a36 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:32.522653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e21888-fc56-4f77-b5f3-98db99069a36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:32.522954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e21888-fc56-4f77-b5f3-98db99069a36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:32.524293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e21888-fc56-4f77-b5f3-98db99069a36 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:32.531981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 637/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:16:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:32.538478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbbd5a-5bf8-4377-adf7-b76050ed11e5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:32.617135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:32.639511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbbd5a-5bf8-4377-adf7-b76050ed11e5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:32.673831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f622b-e43a-4ee8-be31-824d96df8c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:32.674450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f622b-e43a-4ee8-be31-824d96df8c29 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:32.674727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f622b-e43a-4ee8-be31-824d96df8c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:32.675001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f622b-e43a-4ee8-be31-824d96df8c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:32.686526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f622b-e43a-4ee8-be31-824d96df8c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:32.689366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f622b-e43a-4ee8-be31-824d96df8c29 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d91f622b-e43a-4ee8-be31-824d96df8c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:32.703274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:32.704397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:32.705679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:32.705679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:32.705679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:32.705679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 614/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:16:32.710730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e21888-fc56-4f77-b5f3-98db99069a36 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:32.822630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e21888-fc56-4f77-b5f3-98db99069a36 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:32.857531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9276bfd-3c5a-4bfa-b56d-d0b38274bb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:32.858168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9276bfd-3c5a-4bfa-b56d-d0b38274bb30 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:32.858395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9276bfd-3c5a-4bfa-b56d-d0b38274bb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:32.858661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9276bfd-3c5a-4bfa-b56d-d0b38274bb30 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:32.859338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9276bfd-3c5a-4bfa-b56d-d0b38274bb30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:32.859629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9276bfd-3c5a-4bfa-b56d-d0b38274bb30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:32.860970 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9276bfd-3c5a-4bfa-b56d-d0b38274bb30 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:32.872848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 638/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:16:32.878554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f622b-e43a-4ee8-be31-824d96df8c29 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:32.980005 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f622b-e43a-4ee8-be31-824d96df8c29 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:33.006602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:33.014932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff39e0-8e50-4a02-8606-d9510dda4f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:33.015613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff39e0-8e50-4a02-8606-d9510dda4f89 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:33.015894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff39e0-8e50-4a02-8606-d9510dda4f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:33.016164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff39e0-8e50-4a02-8606-d9510dda4f89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:33.023700 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff39e0-8e50-4a02-8606-d9510dda4f89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:33.025149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff39e0-8e50-4a02-8606-d9510dda4f89 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-c6ff39e0-8e50-4a02-8606-d9510dda4f89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:16:33.035860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:33.041812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:33.043106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 615/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:33.043900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9276bfd-3c5a-4bfa-b56d-d0b38274bb30 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:33.099697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:33.143978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9276bfd-3c5a-4bfa-b56d-d0b38274bb30 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:33.178672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47f9ac-8257-4c96-808b-61289b4bc4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:33.179278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47f9ac-8257-4c96-808b-61289b4bc4fd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:33.179556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47f9ac-8257-4c96-808b-61289b4bc4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:33.179826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47f9ac-8257-4c96-808b-61289b4bc4fd None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:33.180543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47f9ac-8257-4c96-808b-61289b4bc4fd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:33.180844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47f9ac-8257-4c96-808b-61289b4bc4fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:33.182320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47f9ac-8257-4c96-808b-61289b4bc4fd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:33.200310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 639/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 1 05:16:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:16:33.206454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff39e0-8e50-4a02-8606-d9510dda4f89 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:33.307292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff39e0-8e50-4a02-8606-d9510dda4f89 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:33.341474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6711579-4a1b-4c47-8a95-f778b124908b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:33.342088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6711579-4a1b-4c47-8a95-f778b124908b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:33.342386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6711579-4a1b-4c47-8a95-f778b124908b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:33.342677 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6711579-4a1b-4c47-8a95-f778b124908b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:33.397823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:33.409841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6711579-4a1b-4c47-8a95-f778b124908b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:33.411292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6711579-4a1b-4c47-8a95-f778b124908b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:33.512659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 616/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:16:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:16:33.518599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47f9ac-8257-4c96-808b-61289b4bc4fd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:33.619705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47f9ac-8257-4c96-808b-61289b4bc4fd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:33.634474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:33.655989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545028e-4f21-4834-9c3e-0b481591563c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:33.656554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545028e-4f21-4834-9c3e-0b481591563c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:33.656834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545028e-4f21-4834-9c3e-0b481591563c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:33.657102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545028e-4f21-4834-9c3e-0b481591563c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:33.666732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545028e-4f21-4834-9c3e-0b481591563c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:33.668174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545028e-4f21-4834-9c3e-0b481591563c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6545028e-4f21-4834-9c3e-0b481591563c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:33.682174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:33.683963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:33.684902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:33.684902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:33.684902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:33.684902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 640/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:16:33.689232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6711579-4a1b-4c47-8a95-f778b124908b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:33.777679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:33.789758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6711579-4a1b-4c47-8a95-f778b124908b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:33.823836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc962198-c393-4354-93d1-bfcff9c77e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:33.824481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc962198-c393-4354-93d1-bfcff9c77e69 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:33.824758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc962198-c393-4354-93d1-bfcff9c77e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:33.825031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc962198-c393-4354-93d1-bfcff9c77e69 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:33.825706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc962198-c393-4354-93d1-bfcff9c77e69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:33.826012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc962198-c393-4354-93d1-bfcff9c77e69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:33.827395 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc962198-c393-4354-93d1-bfcff9c77e69 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:33.840132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 617/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:33.845538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545028e-4f21-4834-9c3e-0b481591563c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:33.943706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545028e-4f21-4834-9c3e-0b481591563c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:33.977907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39754cf9-9320-48bf-a182-a6abf682e381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:33.980417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39754cf9-9320-48bf-a182-a6abf682e381 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:33.980417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39754cf9-9320-48bf-a182-a6abf682e381 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:33.980417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39754cf9-9320-48bf-a182-a6abf682e381 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:33.985764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39754cf9-9320-48bf-a182-a6abf682e381 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:33.987193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39754cf9-9320-48bf-a182-a6abf682e381 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-39754cf9-9320-48bf-a182-a6abf682e381 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:16:33.998433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:33.999477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:34.000753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:16:34.000753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:16:34.000753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:16:34.000753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:16:34.000753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:16:34.000753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:16:34.000753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:34.000753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:34.000753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 641/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:34.005390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc962198-c393-4354-93d1-bfcff9c77e69 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:34.107246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc962198-c393-4354-93d1-bfcff9c77e69 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:34.139849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adace657-4c92-4610-8505-14c830f817a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:34.140450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adace657-4c92-4610-8505-14c830f817a6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:34.140730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adace657-4c92-4610-8505-14c830f817a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:34.141002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adace657-4c92-4610-8505-14c830f817a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:34.141719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adace657-4c92-4610-8505-14c830f817a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:34.142026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adace657-4c92-4610-8505-14c830f817a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:34.143456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adace657-4c92-4610-8505-14c830f817a6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:34.156119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 618/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:34.163010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39754cf9-9320-48bf-a182-a6abf682e381 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:34.188979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:34.263621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39754cf9-9320-48bf-a182-a6abf682e381 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:34.298699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bae89-5aa4-4b7e-a0ed-b3057cb02d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:34.299307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bae89-5aa4-4b7e-a0ed-b3057cb02d61 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:34.299587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bae89-5aa4-4b7e-a0ed-b3057cb02d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:34.299862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bae89-5aa4-4b7e-a0ed-b3057cb02d61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:34.300581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bae89-5aa4-4b7e-a0ed-b3057cb02d61 None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=c11c7fb612ab40d88f526b8b38dbfb23) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:34.300889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bae89-5aa4-4b7e-a0ed-b3057cb02d61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:34.302388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bae89-5aa4-4b7e-a0ed-b3057cb02d61 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:34.307694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 642/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:16:34 2022] GET /identity/v3/role_assignments?user.id=c11c7fb612ab40d88f526b8b38dbfb23&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:16:36.288407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c589a-38e7-4178-8152-6b739acf49dc None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:36.288964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c589a-38e7-4178-8152-6b739acf49dc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:36.288964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c589a-38e7-4178-8152-6b739acf49dc None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:36.290441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 619/1261] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:36.295672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40335e-b109-4a22-9904-97fac9c05131 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:36.296078 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40335e-b109-4a22-9904-97fac9c05131 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:36.296225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40335e-b109-4a22-9904-97fac9c05131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:36.314414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2f40335e-b109-4a22-9904-97fac9c05131 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:36.315608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2f40335e-b109-4a22-9904-97fac9c05131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:36.382241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 643/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:16:36.658115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c589a-38e7-4178-8152-6b739acf49dc None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:36.669374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:36.761536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c589a-38e7-4178-8152-6b739acf49dc None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:36.797117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15028d3-7df8-4439-a66d-7bd72e2daa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:36.797779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15028d3-7df8-4439-a66d-7bd72e2daa5c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:36.798061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15028d3-7df8-4439-a66d-7bd72e2daa5c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:36.798333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15028d3-7df8-4439-a66d-7bd72e2daa5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:36.808606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15028d3-7df8-4439-a66d-7bd72e2daa5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e15028d3-7df8-4439-a66d-7bd72e2daa5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:16:36.813339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:36.818773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:36.819820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 620/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:36.821523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40335e-b109-4a22-9904-97fac9c05131 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:36.840120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:36.919812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40335e-b109-4a22-9904-97fac9c05131 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:36.955136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22344ffd-ca58-4f63-b991-7bfff9957b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:36.955745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22344ffd-ca58-4f63-b991-7bfff9957b91 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:36.956025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22344ffd-ca58-4f63-b991-7bfff9957b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:36.956298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22344ffd-ca58-4f63-b991-7bfff9957b91 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:36.956962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22344ffd-ca58-4f63-b991-7bfff9957b91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:36.957266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22344ffd-ca58-4f63-b991-7bfff9957b91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:36.958694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22344ffd-ca58-4f63-b991-7bfff9957b91 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:36.966971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 644/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:36.972768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15028d3-7df8-4439-a66d-7bd72e2daa5c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:37.039993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:37.073657 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15028d3-7df8-4439-a66d-7bd72e2daa5c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:37.106498 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794458-9892-42a0-81cb-655d37167bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:37.107109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794458-9892-42a0-81cb-655d37167bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:37.107385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794458-9892-42a0-81cb-655d37167bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:37.107656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794458-9892-42a0-81cb-655d37167bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:37.118003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66794458-9892-42a0-81cb-655d37167bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:37.119448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66794458-9892-42a0-81cb-655d37167bb5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-66794458-9892-42a0-81cb-655d37167bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:37.134793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:37.135986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:37.137111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:37.137111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:37.137111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:37.137111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 621/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:37.141849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22344ffd-ca58-4f63-b991-7bfff9957b91 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:37.225609 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:37.245752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22344ffd-ca58-4f63-b991-7bfff9957b91 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:37.278369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955f848-237f-4c15-a11e-91268b277981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:37.279024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955f848-237f-4c15-a11e-91268b277981 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:37.279303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955f848-237f-4c15-a11e-91268b277981 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:37.279572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955f848-237f-4c15-a11e-91268b277981 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:37.280259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955f848-237f-4c15-a11e-91268b277981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:37.280559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955f848-237f-4c15-a11e-91268b277981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:37.282040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955f848-237f-4c15-a11e-91268b277981 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:37.294027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 645/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:37.299926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794458-9892-42a0-81cb-655d37167bb5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:37.401714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794458-9892-42a0-81cb-655d37167bb5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:37.422558 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:37.437114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa91174-4627-4e27-a76e-7d381ccdc9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:37.437773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa91174-4627-4e27-a76e-7d381ccdc9bf None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:37.438053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa91174-4627-4e27-a76e-7d381ccdc9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:37.438324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa91174-4627-4e27-a76e-7d381ccdc9bf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:37.446493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa91174-4627-4e27-a76e-7d381ccdc9bf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:37.447981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa91174-4627-4e27-a76e-7d381ccdc9bf None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-7aa91174-4627-4e27-a76e-7d381ccdc9bf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:16:37.458962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:37.459940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:37.460908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:37.460908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:37.460908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:16:37.460908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:16:37.460908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:16:37.460908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:16:37.460908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:16:37.460908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:16:37.460908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:37.460908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:37.461530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 622/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:37.466066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955f848-237f-4c15-a11e-91268b277981 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:37.565231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955f848-237f-4c15-a11e-91268b277981 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:37.598478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2135d0a-c497-46e9-80b4-45f17e1ddc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:37.599711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2135d0a-c497-46e9-80b4-45f17e1ddc74 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:37.599986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2135d0a-c497-46e9-80b4-45f17e1ddc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:37.600251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2135d0a-c497-46e9-80b4-45f17e1ddc74 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:37.601051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2135d0a-c497-46e9-80b4-45f17e1ddc74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:37.601353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2135d0a-c497-46e9-80b4-45f17e1ddc74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:37.602787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2135d0a-c497-46e9-80b4-45f17e1ddc74 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:37.611169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:37.620133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 646/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 1 05:16:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:37.625904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa91174-4627-4e27-a76e-7d381ccdc9bf None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:37.727436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa91174-4627-4e27-a76e-7d381ccdc9bf None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:37.760690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dbaeb-fe08-4ed4-abb6-0fec846f3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:37.761309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dbaeb-fe08-4ed4-abb6-0fec846f3992 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:37.761686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dbaeb-fe08-4ed4-abb6-0fec846f3992 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:37.761955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dbaeb-fe08-4ed4-abb6-0fec846f3992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:37.772788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1dbaeb-fe08-4ed4-abb6-0fec846f3992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:37.774283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1dbaeb-fe08-4ed4-abb6-0fec846f3992 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-dd1dbaeb-fe08-4ed4-abb6-0fec846f3992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:37.788835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:37.795028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:37.795961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:37.795961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:37.795961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:37.795961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 623/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:37.796231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2135d0a-c497-46e9-80b4-45f17e1ddc74 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:37.893822 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2135d0a-c497-46e9-80b4-45f17e1ddc74 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:37.927895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26754caa-cc43-4d63-805b-ad357ecfde15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:37.928529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26754caa-cc43-4d63-805b-ad357ecfde15 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:37.928813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26754caa-cc43-4d63-805b-ad357ecfde15 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:37.929103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26754caa-cc43-4d63-805b-ad357ecfde15 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:37.929777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26754caa-cc43-4d63-805b-ad357ecfde15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:37.930083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26754caa-cc43-4d63-805b-ad357ecfde15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:37.931489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26754caa-cc43-4d63-805b-ad357ecfde15 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:37.943248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 647/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:37.948928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dbaeb-fe08-4ed4-abb6-0fec846f3992 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:37.981802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:38.050103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dbaeb-fe08-4ed4-abb6-0fec846f3992 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:38.084290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322089c-3519-4562-b037-0f32023b87ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:38.084905 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322089c-3519-4562-b037-0f32023b87ac None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:38.085186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322089c-3519-4562-b037-0f32023b87ac None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:38.085483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322089c-3519-4562-b037-0f32023b87ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:38.091890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322089c-3519-4562-b037-0f32023b87ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:38.093327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322089c-3519-4562-b037-0f32023b87ac None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-1322089c-3519-4562-b037-0f32023b87ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:16:38.104481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:38.105644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:38.106786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:16:38.106786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:16:38.106786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:16:38.106786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:16:38.106786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:16:38.106786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:16:38.106786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:38.106786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:38.106786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 624/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:16:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:38.111628 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26754caa-cc43-4d63-805b-ad357ecfde15 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:38.169899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:38.212145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26754caa-cc43-4d63-805b-ad357ecfde15 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:38.244934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34844149-764e-404b-87bb-03f310e231dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:38.245577 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34844149-764e-404b-87bb-03f310e231dd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:38.245859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34844149-764e-404b-87bb-03f310e231dd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:38.246134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34844149-764e-404b-87bb-03f310e231dd None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:38.246825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34844149-764e-404b-87bb-03f310e231dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:38.247184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34844149-764e-404b-87bb-03f310e231dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:38.248753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34844149-764e-404b-87bb-03f310e231dd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:38.262033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 648/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:38.268240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322089c-3519-4562-b037-0f32023b87ac None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:38.360144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:38.369610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322089c-3519-4562-b037-0f32023b87ac None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:38.402668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feff717-29e1-4fdb-a1a4-d02c1a44f948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:38.403314 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff717-29e1-4fdb-a1a4-d02c1a44f948 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:38.403595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff717-29e1-4fdb-a1a4-d02c1a44f948 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:38.403868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff717-29e1-4fdb-a1a4-d02c1a44f948 None admin] PATH_INFO: `/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/c11c7fb612ab40d88f526b8b38dbfb23/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:38.405816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feff717-29e1-4fdb-a1a4-d02c1a44f948 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4b03d9fda4f49cb92fe52745e17e5bf, user_id=c11c7fb612ab40d88f526b8b38dbfb23, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:38.407335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feff717-29e1-4fdb-a1a4-d02c1a44f948 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:38.466009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 625/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 1 05:16:38 2022] PUT /identity/v3/projects/a4b03d9fda4f49cb92fe52745e17e5bf/users/c11c7fb612ab40d88f526b8b38dbfb23/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:16:40.377469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420729ef-3108-414c-93cd-1628f7368886 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:40.378036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420729ef-3108-414c-93cd-1628f7368886 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:40.378036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420729ef-3108-414c-93cd-1628f7368886 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:40.379358 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 649/1274] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:40.385011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0d552-425e-4781-990a-9a661e529e9d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:40.389147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0d552-425e-4781-990a-9a661e529e9d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:40.389147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0d552-425e-4781-990a-9a661e529e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:40.404845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-bea0d552-425e-4781-990a-9a661e529e9d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:40.406070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bea0d552-425e-4781-990a-9a661e529e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:40.472463 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 626/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:16:40.743566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420729ef-3108-414c-93cd-1628f7368886 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:40.852068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420729ef-3108-414c-93cd-1628f7368886 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:40.869148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:40.886079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23834b94-0351-4635-8710-b095acff7c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:40.886703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23834b94-0351-4635-8710-b095acff7c65 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:40.887063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23834b94-0351-4635-8710-b095acff7c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:40.887335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23834b94-0351-4635-8710-b095acff7c65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:40.897040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23834b94-0351-4635-8710-b095acff7c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-23834b94-0351-4635-8710-b095acff7c65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:16:40.901911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:40.903132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:40.904019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 650/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 1 05:16:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:40.909218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0d552-425e-4781-990a-9a661e529e9d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:40.990402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:41.008476 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0d552-425e-4781-990a-9a661e529e9d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:41.041900 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41345c0c-663c-4bb5-af8e-8e38d558b3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:41.042503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41345c0c-663c-4bb5-af8e-8e38d558b3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:41.042781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41345c0c-663c-4bb5-af8e-8e38d558b3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:41.043053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41345c0c-663c-4bb5-af8e-8e38d558b3a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:41.043709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41345c0c-663c-4bb5-af8e-8e38d558b3a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:41.044011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41345c0c-663c-4bb5-af8e-8e38d558b3a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:41.045369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41345c0c-663c-4bb5-af8e-8e38d558b3a3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:41.054007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 627/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 1 05:16:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:41.059849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23834b94-0351-4635-8710-b095acff7c65 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:41.162052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23834b94-0351-4635-8710-b095acff7c65 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:41.196282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b654a5f-49a2-4fd5-a30a-03210fa1e90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:41.196896 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b654a5f-49a2-4fd5-a30a-03210fa1e90b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:41.197172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b654a5f-49a2-4fd5-a30a-03210fa1e90b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:41.197441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b654a5f-49a2-4fd5-a30a-03210fa1e90b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:41.207365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b654a5f-49a2-4fd5-a30a-03210fa1e90b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:41.209372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b654a5f-49a2-4fd5-a30a-03210fa1e90b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3b654a5f-49a2-4fd5-a30a-03210fa1e90b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:41.223538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:41.224710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:41.225650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:41.225650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:41.225650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:41.225650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 651/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:41.231051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41345c0c-663c-4bb5-af8e-8e38d558b3a3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:41.330828 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41345c0c-663c-4bb5-af8e-8e38d558b3a3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:41.363926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101570c-7df5-4a55-96cb-8c8afd02c3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:41.365101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101570c-7df5-4a55-96cb-8c8afd02c3bb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:41.365376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101570c-7df5-4a55-96cb-8c8afd02c3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:41.365671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101570c-7df5-4a55-96cb-8c8afd02c3bb None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:41.366305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101570c-7df5-4a55-96cb-8c8afd02c3bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:41.366605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101570c-7df5-4a55-96cb-8c8afd02c3bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:41.367967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101570c-7df5-4a55-96cb-8c8afd02c3bb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:41.379555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 628/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:41.385892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b654a5f-49a2-4fd5-a30a-03210fa1e90b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:41.416202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:41.485767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b654a5f-49a2-4fd5-a30a-03210fa1e90b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:41.519518 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348bb57-2a4d-4fe6-87a5-e3dd31aeb98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:41.520146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348bb57-2a4d-4fe6-87a5-e3dd31aeb98b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:41.520540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348bb57-2a4d-4fe6-87a5-e3dd31aeb98b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:41.520976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348bb57-2a4d-4fe6-87a5-e3dd31aeb98b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:41.528327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348bb57-2a4d-4fe6-87a5-e3dd31aeb98b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:41.529790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348bb57-2a4d-4fe6-87a5-e3dd31aeb98b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-b348bb57-2a4d-4fe6-87a5-e3dd31aeb98b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:16:41.541079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:41.542105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:41.543193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 652/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:41.548069 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101570c-7df5-4a55-96cb-8c8afd02c3bb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:41.552686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:41.647171 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101570c-7df5-4a55-96cb-8c8afd02c3bb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:41.681388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d2291-a645-4377-81da-55028bdffbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:41.682024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2291-a645-4377-81da-55028bdffbdb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:41.682300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2291-a645-4377-81da-55028bdffbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:41.682570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2291-a645-4377-81da-55028bdffbdb None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:41.683293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d2291-a645-4377-81da-55028bdffbdb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:41.683600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d2291-a645-4377-81da-55028bdffbdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:41.685030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d2291-a645-4377-81da-55028bdffbdb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:41.703475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 629/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 1 05:16:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:41.709378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348bb57-2a4d-4fe6-87a5-e3dd31aeb98b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:41.799819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:41.810094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348bb57-2a4d-4fe6-87a5-e3dd31aeb98b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:41.843022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b484f4e-d830-4108-aee6-d675c2cb28c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:41.843619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b484f4e-d830-4108-aee6-d675c2cb28c0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:41.843892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b484f4e-d830-4108-aee6-d675c2cb28c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:41.844272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b484f4e-d830-4108-aee6-d675c2cb28c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:41.909698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b484f4e-d830-4108-aee6-d675c2cb28c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:41.911736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b484f4e-d830-4108-aee6-d675c2cb28c0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:41.998365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:42.008382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 653/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 1 05:16:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:16:42.015954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d2291-a645-4377-81da-55028bdffbdb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:42.082661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:42.122354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d2291-a645-4377-81da-55028bdffbdb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:42.155895 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f3f94-fd2b-4bd9-90e6-a650193bc4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:42.156501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f3f94-fd2b-4bd9-90e6-a650193bc4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:42.156778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f3f94-fd2b-4bd9-90e6-a650193bc4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:42.157045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f3f94-fd2b-4bd9-90e6-a650193bc4d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:42.167768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f3f94-fd2b-4bd9-90e6-a650193bc4d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:42.169148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f3f94-fd2b-4bd9-90e6-a650193bc4d7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-ac7f3f94-fd2b-4bd9-90e6-a650193bc4d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:16:42.183986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:42.185154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:16:42.186322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:16:42.186322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:16:42.186322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:42.186322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 630/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:42.191300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b484f4e-d830-4108-aee6-d675c2cb28c0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:42.288828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b484f4e-d830-4108-aee6-d675c2cb28c0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:42.324049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32313e36-9033-469c-abce-45139077108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:42.324690 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32313e36-9033-469c-abce-45139077108c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:42.324975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32313e36-9033-469c-abce-45139077108c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:42.325245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32313e36-9033-469c-abce-45139077108c None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:42.325975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32313e36-9033-469c-abce-45139077108c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:42.326487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32313e36-9033-469c-abce-45139077108c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:42.328944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32313e36-9033-469c-abce-45139077108c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:42.344088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 654/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 1 05:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:42.350130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f3f94-fd2b-4bd9-90e6-a650193bc4d7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:42.449988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f3f94-fd2b-4bd9-90e6-a650193bc4d7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:42.463854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:42.482999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4b63b-170f-4cba-866b-0eaebffe62a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:42.483595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4b63b-170f-4cba-866b-0eaebffe62a3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:42.483870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4b63b-170f-4cba-866b-0eaebffe62a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:42.484134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4b63b-170f-4cba-866b-0eaebffe62a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:42.491719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4b63b-170f-4cba-866b-0eaebffe62a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:42.494274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4b63b-170f-4cba-866b-0eaebffe62a3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-c5b4b63b-170f-4cba-866b-0eaebffe62a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:16:42.504238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:42.505993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:42.506976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:16:42.506976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:16:42.506976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:16:42.506976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:16:42.506976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:16:42.506976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:16:42.506976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:16:42.506976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 01 05:16:42.506976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 631/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 1 05:16:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:42.512018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32313e36-9033-469c-abce-45139077108c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:42.561732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:42.610196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32313e36-9033-469c-abce-45139077108c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:42.643728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92ad16-9265-4dd9-b746-445421b65c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:42.644327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92ad16-9265-4dd9-b746-445421b65c89 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:42.644602 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92ad16-9265-4dd9-b746-445421b65c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:42.644871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92ad16-9265-4dd9-b746-445421b65c89 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:42.645614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92ad16-9265-4dd9-b746-445421b65c89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:42.645918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92ad16-9265-4dd9-b746-445421b65c89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:42.647357 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92ad16-9265-4dd9-b746-445421b65c89 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:42.659004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 655/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 1 05:16:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:42.665731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4b63b-170f-4cba-866b-0eaebffe62a3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:42.766712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4b63b-170f-4cba-866b-0eaebffe62a3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:42.799104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde4ff2-dae2-4dfb-9f56-1306625fbcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:42.799687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde4ff2-dae2-4dfb-9f56-1306625fbcaa None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:42.799960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde4ff2-dae2-4dfb-9f56-1306625fbcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:42.800222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde4ff2-dae2-4dfb-9f56-1306625fbcaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:42.800931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde4ff2-dae2-4dfb-9f56-1306625fbcaa None admin] RBAC: Adding query filter params (role.id=4bd9fea9bc2648c2ada62148466f2700, scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf, user.id=c11c7fb612ab40d88f526b8b38dbfb23) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:42.801230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde4ff2-dae2-4dfb-9f56-1306625fbcaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:42.803318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde4ff2-dae2-4dfb-9f56-1306625fbcaa None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:42.808896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 632/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 1 05:16:42 2022] GET /identity/v3/role_assignments?user.id=c11c7fb612ab40d88f526b8b38dbfb23&scope.project.id=a4b03d9fda4f49cb92fe52745e17e5bf&role.id=4bd9fea9bc2648c2ada62148466f2700 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:44.747467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f237c79-5195-41a7-bc4f-053f4ed7829e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:44.747917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f237c79-5195-41a7-bc4f-053f4ed7829e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:44.748026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f237c79-5195-41a7-bc4f-053f4ed7829e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:44.749351 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 656/1288] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:44.755119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335fb8e-251b-4516-9b4c-0c101054cbb0 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:44.755604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335fb8e-251b-4516-9b4c-0c101054cbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:44.755777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335fb8e-251b-4516-9b4c-0c101054cbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:44.774425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6335fb8e-251b-4516-9b4c-0c101054cbb0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:44.775623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6335fb8e-251b-4516-9b4c-0c101054cbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:44.805646 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:44.845678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 633/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:16:45.111160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f237c79-5195-41a7-bc4f-053f4ed7829e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:45.213053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f237c79-5195-41a7-bc4f-053f4ed7829e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:45.223721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:45.247201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b4ee5-739c-4bd1-8b45-80f5b3ac61d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:45.247828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b4ee5-739c-4bd1-8b45-80f5b3ac61d4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:45.248114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b4ee5-739c-4bd1-8b45-80f5b3ac61d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:45.248389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b4ee5-739c-4bd1-8b45-80f5b3ac61d4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:45.249011 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b4ee5-739c-4bd1-8b45-80f5b3ac61d4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:45.250499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b4ee5-739c-4bd1-8b45-80f5b3ac61d4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-627b4ee5-739c-4bd1-8b45-80f5b3ac61d4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:45.260863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:45.262086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:45.262965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 657/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 1 05:16:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:45.268207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335fb8e-251b-4516-9b4c-0c101054cbb0 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:45.367541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335fb8e-251b-4516-9b4c-0c101054cbb0 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:45.401103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654caa09-1d6f-4211-a0d1-fe6c634e021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:45.401756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654caa09-1d6f-4211-a0d1-fe6c634e021b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:45.402035 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654caa09-1d6f-4211-a0d1-fe6c634e021b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:45.402306 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654caa09-1d6f-4211-a0d1-fe6c634e021b None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:45.402927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654caa09-1d6f-4211-a0d1-fe6c634e021b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:45.403226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654caa09-1d6f-4211-a0d1-fe6c634e021b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:45.404591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654caa09-1d6f-4211-a0d1-fe6c634e021b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:45.409355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 634/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 1 05:16:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:45.416124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b4ee5-739c-4bd1-8b45-80f5b3ac61d4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:45.517673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b4ee5-739c-4bd1-8b45-80f5b3ac61d4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:45.552114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc27a9-f6fa-487d-ac65-d2a17512a991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:45.552819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc27a9-f6fa-487d-ac65-d2a17512a991 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:45.553097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc27a9-f6fa-487d-ac65-d2a17512a991 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:45.553362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc27a9-f6fa-487d-ac65-d2a17512a991 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:45.554010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc27a9-f6fa-487d-ac65-d2a17512a991 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:45.554378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc27a9-f6fa-487d-ac65-d2a17512a991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:45.555728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc27a9-f6fa-487d-ac65-d2a17512a991 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:45.560139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 658/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 1 05:16:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:47.479709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07379577-2072-4118-9a77-dcf3d8922c4a None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:47.480244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07379577-2072-4118-9a77-dcf3d8922c4a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:47.480244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07379577-2072-4118-9a77-dcf3d8922c4a None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:47.481601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 635/1293] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:47.487056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836c132-b12b-4e3d-a562-05bf2ab71505 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:47.487465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836c132-b12b-4e3d-a562-05bf2ab71505 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:47.487599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836c132-b12b-4e3d-a562-05bf2ab71505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:47.506671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9836c132-b12b-4e3d-a562-05bf2ab71505 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:47.507890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9836c132-b12b-4e3d-a562-05bf2ab71505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:47.538626 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:47.575336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 659/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:16:47.855980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07379577-2072-4118-9a77-dcf3d8922c4a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:47.887730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:47.958809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07379577-2072-4118-9a77-dcf3d8922c4a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:47.993774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb4579-355a-43e6-9b96-3dace472ee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:47.994369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb4579-355a-43e6-9b96-3dace472ee25 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:47.994643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb4579-355a-43e6-9b96-3dace472ee25 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:47.994911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb4579-355a-43e6-9b96-3dace472ee25 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:47.995592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb4579-355a-43e6-9b96-3dace472ee25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:47.996945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb4579-355a-43e6-9b96-3dace472ee25 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:48.059317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 636/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 1 05:16:47 2022] POST /identity/v3/services => generated 246 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:16:50.001329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6a2d4-3ebe-4450-a41a-a3ed85d10254 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:50.001869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6a2d4-3ebe-4450-a41a-a3ed85d10254 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:50.001869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6a2d4-3ebe-4450-a41a-a3ed85d10254 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:50.003152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 660/1296] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:50.009030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2797f63-3da0-4ed9-8ba1-d23dacf72efa None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:50.009495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2797f63-3da0-4ed9-8ba1-d23dacf72efa None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:50.009592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2797f63-3da0-4ed9-8ba1-d23dacf72efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:50.027945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d2797f63-3da0-4ed9-8ba1-d23dacf72efa None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:50.029214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d2797f63-3da0-4ed9-8ba1-d23dacf72efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:50.103157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 637/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:16:50.377101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6a2d4-3ebe-4450-a41a-a3ed85d10254 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:50.478045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6a2d4-3ebe-4450-a41a-a3ed85d10254 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:50.511932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c386b-9f11-4dba-892a-41338e53bcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:50.512563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c386b-9f11-4dba-892a-41338e53bcce None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:50.512840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c386b-9f11-4dba-892a-41338e53bcce None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:50.513107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c386b-9f11-4dba-892a-41338e53bcce None admin] PATH_INFO: `/v3/services/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:50.513716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c386b-9f11-4dba-892a-41338e53bcce None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:50.515069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c386b-9f11-4dba-892a-41338e53bcce None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c38c386b-9f11-4dba-892a-41338e53bcce None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:50.524372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:50.525548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:50.526649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 661/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 1 05:16:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:50.531469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2797f63-3da0-4ed9-8ba1-d23dacf72efa None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:50.550010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:50.643956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2797f63-3da0-4ed9-8ba1-d23dacf72efa None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:50.678910 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464ab30-de73-46fe-bc93-321796946f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:50.679508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464ab30-de73-46fe-bc93-321796946f61 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:50.679782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464ab30-de73-46fe-bc93-321796946f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:50.680050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464ab30-de73-46fe-bc93-321796946f61 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:50.680756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464ab30-de73-46fe-bc93-321796946f61 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:50.681061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464ab30-de73-46fe-bc93-321796946f61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:50.682455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464ab30-de73-46fe-bc93-321796946f61 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:50.687454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 638/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 1 05:16:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:50.693852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c386b-9f11-4dba-892a-41338e53bcce None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:50.726766 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:50.794590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c386b-9f11-4dba-892a-41338e53bcce None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:50.827855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d12f9-2d04-4c5c-9d75-9f7202168254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:50.828457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d12f9-2d04-4c5c-9d75-9f7202168254 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:50.828737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d12f9-2d04-4c5c-9d75-9f7202168254 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:50.829007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d12f9-2d04-4c5c-9d75-9f7202168254 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:50.829785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d12f9-2d04-4c5c-9d75-9f7202168254 None admin] RBAC: Adding query filter params (interface=public, service_id=db7bc1466ada4fa9a4333eec9a6426c1, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:50.830092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d12f9-2d04-4c5c-9d75-9f7202168254 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:50.831440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d12f9-2d04-4c5c-9d75-9f7202168254 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:50.837304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 662/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 1 05:16:50 2022] GET /identity/v3/endpoints?service_id=db7bc1466ada4fa9a4333eec9a6426c1&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:50.842973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464ab30-de73-46fe-bc93-321796946f61 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:50.927144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:50.944943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464ab30-de73-46fe-bc93-321796946f61 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:50.978561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ede0a1-45c7-470c-8487-e62467c14d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:50.979161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ede0a1-45c7-470c-8487-e62467c14d4d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:50.979440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ede0a1-45c7-470c-8487-e62467c14d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:50.979715 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ede0a1-45c7-470c-8487-e62467c14d4d None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:50.980358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ede0a1-45c7-470c-8487-e62467c14d4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:50.981752 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ede0a1-45c7-470c-8487-e62467c14d4d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:50.990086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 639/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 1 05:16:50 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:16:52.912206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375312e-d5c7-4338-95de-d4601622f4cb None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:52.912779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375312e-d5c7-4338-95de-d4601622f4cb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:52.912779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375312e-d5c7-4338-95de-d4601622f4cb None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:52.914116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 663/1302] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:16:52.919283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91b8e6-55e8-4cff-89d4-dff302691f2e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:52.919674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91b8e6-55e8-4cff-89d4-dff302691f2e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:52.919831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91b8e6-55e8-4cff-89d4-dff302691f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:52.937951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5c91b8e6-55e8-4cff-89d4-dff302691f2e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:16:52.939129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5c91b8e6-55e8-4cff-89d4-dff302691f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:16:53.007772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 640/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:16:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:16:53.275848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375312e-d5c7-4338-95de-d4601622f4cb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:53.371284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:53.377862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375312e-d5c7-4338-95de-d4601622f4cb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:53.410784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c17cd-d75f-4cbd-8481-437a133ad4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:53.411412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c17cd-d75f-4cbd-8481-437a133ad4df None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:53.411747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c17cd-d75f-4cbd-8481-437a133ad4df None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:53.412229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c17cd-d75f-4cbd-8481-437a133ad4df None admin] PATH_INFO: `/v3/services/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:53.413059 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c17cd-d75f-4cbd-8481-437a133ad4df None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:53.414656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c17cd-d75f-4cbd-8481-437a133ad4df None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-748c17cd-d75f-4cbd-8481-437a133ad4df None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:16:53.423810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:16:53.424975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 01 05:16:53.426867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 664/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 1 05:16:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:16:53.431531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91b8e6-55e8-4cff-89d4-dff302691f2e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:53.508449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:16:53.530570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91b8e6-55e8-4cff-89d4-dff302691f2e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:53.566116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70f9c7-64f7-4a7d-9c94-98f0f7bf5b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:53.567718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70f9c7-64f7-4a7d-9c94-98f0f7bf5b03 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:53.568095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70f9c7-64f7-4a7d-9c94-98f0f7bf5b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:53.568369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70f9c7-64f7-4a7d-9c94-98f0f7bf5b03 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:53.568991 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70f9c7-64f7-4a7d-9c94-98f0f7bf5b03 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:16:53.569289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70f9c7-64f7-4a7d-9c94-98f0f7bf5b03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:53.570675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70f9c7-64f7-4a7d-9c94-98f0f7bf5b03 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:53.575972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 641/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 1 05:16:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:16:53.582147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c17cd-d75f-4cbd-8481-437a133ad4df None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:16:53.683709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c17cd-d75f-4cbd-8481-437a133ad4df None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:16:53.717083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf4b2f-7f01-4110-8a7d-09c927daf540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:16:53.717717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4b2f-7f01-4110-8a7d-09c927daf540 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:16:53.717993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4b2f-7f01-4110-8a7d-09c927daf540 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:16:53.718261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4b2f-7f01-4110-8a7d-09c927daf540 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:16:53.718915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf4b2f-7f01-4110-8a7d-09c927daf540 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:16:53.720361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf4b2f-7f01-4110-8a7d-09c927daf540 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:16:53.803013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 665/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:16:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:17:03.573239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=30 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:17:03.797524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:33.586198 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c46303-1cd1-471c-83e3-f34712bb89e7 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:33.586672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c46303-1cd1-471c-83e3-f34712bb89e7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:33.586900 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c46303-1cd1-471c-83e3-f34712bb89e7 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:33.589504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 642/1307] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:22:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:33.607563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d629048-a2f9-4b4e-847e-b1ee6d209a9b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:33.607563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d629048-a2f9-4b4e-847e-b1ee6d209a9b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:33.607563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d629048-a2f9-4b4e-847e-b1ee6d209a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:33.700512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2d629048-a2f9-4b4e-847e-b1ee6d209a9b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:33.701693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2d629048-a2f9-4b4e-847e-b1ee6d209a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:33.824956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 666/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:34.124831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8658ce-72cc-415a-aa7e-ba0cd3bffa59 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:34.125186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8658ce-72cc-415a-aa7e-ba0cd3bffa59 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:34.125369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8658ce-72cc-415a-aa7e-ba0cd3bffa59 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:34.126901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 643/1309] 10.222.0.42 () {54 vars in 965 bytes} [Thu Sep 1 05:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:34.133416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c593-99e8-4d13-ac64-5a7cd0012cc7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:34.133838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c593-99e8-4d13-ac64-5a7cd0012cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:34.134187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c593-99e8-4d13-ac64-5a7cd0012cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:34.214905 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-59a8c593-99e8-4d13-ac64-5a7cd0012cc7 None None] MFA Rules not processed for user `0e21910fca40479ca04e7f0418f7dd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:34.216033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-59a8c593-99e8-4d13-ac64-5a7cd0012cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:34.239240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:34.306712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 667/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Thu Sep 1 05:22:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:34.316466 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8658ce-72cc-415a-aa7e-ba0cd3bffa59 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:34.318394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bd8658ce-72cc-415a-aa7e-ba0cd3bffa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:34.446605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8658ce-72cc-415a-aa7e-ba0cd3bffa59 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:34.525821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2f099-c6e7-42c3-9eaa-a923ab031bcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:34.526741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2f099-c6e7-42c3-9eaa-a923ab031bcd service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:34.527208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2f099-c6e7-42c3-9eaa-a923ab031bcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:34.527652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2f099-c6e7-42c3-9eaa-a923ab031bcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:34.529420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5bd2f099-c6e7-42c3-9eaa-a923ab031bcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:34.588516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:34.645714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2f099-c6e7-42c3-9eaa-a923ab031bcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:34.647930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2f099-c6e7-42c3-9eaa-a923ab031bcd service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:34.667794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 644/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:36.912062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994409ab-3c29-445c-8ac7-acaecaa570dc None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:36.913164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994409ab-3c29-445c-8ac7-acaecaa570dc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:36.913164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994409ab-3c29-445c-8ac7-acaecaa570dc None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:36.914760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 668/1312] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:36.920181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c068c57-4bac-4cc5-877b-01ad956875a8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:36.920722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c068c57-4bac-4cc5-877b-01ad956875a8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:36.920722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c068c57-4bac-4cc5-877b-01ad956875a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:37.003381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3c068c57-4bac-4cc5-877b-01ad956875a8 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:37.004477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3c068c57-4bac-4cc5-877b-01ad956875a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:37.112119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 645/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:37.398017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994409ab-3c29-445c-8ac7-acaecaa570dc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:37.399559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-994409ab-3c29-445c-8ac7-acaecaa570dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:37.504357 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994409ab-3c29-445c-8ac7-acaecaa570dc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:37.605507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:37.605507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485483f-9b2d-4c1b-8ddc-e97c0fae6d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:37.605507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485483f-9b2d-4c1b-8ddc-e97c0fae6d6e service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:37.605507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485483f-9b2d-4c1b-8ddc-e97c0fae6d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:37.606019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485483f-9b2d-4c1b-8ddc-e97c0fae6d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:37.606019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b485483f-9b2d-4c1b-8ddc-e97c0fae6d6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:37.695008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485483f-9b2d-4c1b-8ddc-e97c0fae6d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:37.696686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485483f-9b2d-4c1b-8ddc-e97c0fae6d6e service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:37.715318 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 669/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:40.130546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a17873-bac3-460d-a7a3-a0757b193ecb None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:40.131423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a17873-bac3-460d-a7a3-a0757b193ecb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:40.131423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a17873-bac3-460d-a7a3-a0757b193ecb None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:40.133291 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 646/1315] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:22:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:40.138916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869afa1-461d-42a8-9264-29b7594ce647 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:40.139359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869afa1-461d-42a8-9264-29b7594ce647 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:40.139459 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869afa1-461d-42a8-9264-29b7594ce647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:40.206476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8869afa1-461d-42a8-9264-29b7594ce647 None None] MFA Rules not processed for user `57a182ce8b7e4cadb8c50cc2a7fa67ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:40.207724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8869afa1-461d-42a8-9264-29b7594ce647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:40.260600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:40.305195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 670/1316] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:40.588335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a17873-bac3-460d-a7a3-a0757b193ecb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:40.589899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-08a17873-bac3-460d-a7a3-a0757b193ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:40.599738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:40.696242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a17873-bac3-460d-a7a3-a0757b193ecb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:40.776576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3fb7e-9acc-40c5-b34c-aad408281d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:40.777178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3fb7e-9acc-40c5-b34c-aad408281d75 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:40.777450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3fb7e-9acc-40c5-b34c-aad408281d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:40.777743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3fb7e-9acc-40c5-b34c-aad408281d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:40.779334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2ca3fb7e-9acc-40c5-b34c-aad408281d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:40.842489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:40.890656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3fb7e-9acc-40c5-b34c-aad408281d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:40.892114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3fb7e-9acc-40c5-b34c-aad408281d75 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:40.910109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 647/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:43.613183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce65a1-7828-4c41-8e77-309affbeaf3f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:43.613779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce65a1-7828-4c41-8e77-309affbeaf3f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:43.614122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce65a1-7828-4c41-8e77-309affbeaf3f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:43.616154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 671/1318] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:22:43.621419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed42f40-57ac-46c3-90a9-36beff44df2b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:43.621909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed42f40-57ac-46c3-90a9-36beff44df2b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:43.622042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed42f40-57ac-46c3-90a9-36beff44df2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:43.695393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4ed42f40-57ac-46c3-90a9-36beff44df2b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:43.696866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4ed42f40-57ac-46c3-90a9-36beff44df2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:43.810106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 648/1319] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:22:44.144863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e783c-7538-4872-b1ae-95c87e22b54c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:44.145145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e783c-7538-4872-b1ae-95c87e22b54c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:44.145415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e783c-7538-4872-b1ae-95c87e22b54c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:44.146939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 672/1320] 10.222.0.42 () {54 vars in 965 bytes} [Thu Sep 1 05:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:44.153058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e4b02-965f-42e3-ad4e-f41fbf174dbb None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:44.153360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e4b02-965f-42e3-ad4e-f41fbf174dbb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:44.153663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e4b02-965f-42e3-ad4e-f41fbf174dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:44.178360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:44.251858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-970e4b02-965f-42e3-ad4e-f41fbf174dbb None None] MFA Rules not processed for user `0e21910fca40479ca04e7f0418f7dd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:44.255238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-970e4b02-965f-42e3-ad4e-f41fbf174dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:44.339033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 649/1321] 10.222.0.42 () {58 vars in 1062 bytes} [Thu Sep 1 05:22:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:44.348102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e783c-7538-4872-b1ae-95c87e22b54c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:44.350180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-865e783c-7538-4872-b1ae-95c87e22b54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:44.454238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e783c-7538-4872-b1ae-95c87e22b54c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:44.526851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f16340-7470-4386-810d-6c4dadcab5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:44.527457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f16340-7470-4386-810d-6c4dadcab5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:44.527741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f16340-7470-4386-810d-6c4dadcab5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:44.528019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f16340-7470-4386-810d-6c4dadcab5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:44.529709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-85f16340-7470-4386-810d-6c4dadcab5f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:44.537868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:44.640393 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f16340-7470-4386-810d-6c4dadcab5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:44.641862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f16340-7470-4386-810d-6c4dadcab5f6 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:44.659500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 673/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:47.982448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb8e6b-7086-45ad-bdd2-463699f96bf2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:47.982981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb8e6b-7086-45ad-bdd2-463699f96bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:47.982981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb8e6b-7086-45ad-bdd2-463699f96bf2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:47.984260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 650/1323] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:47.989592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba37071-139c-4108-bf4c-28407e218f43 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:47.990140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba37071-139c-4108-bf4c-28407e218f43 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:47.990268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba37071-139c-4108-bf4c-28407e218f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:48.057883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-eba37071-139c-4108-bf4c-28407e218f43 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:48.059140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eba37071-139c-4108-bf4c-28407e218f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:48.118500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:48.158664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 674/1324] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:48.451480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb8e6b-7086-45ad-bdd2-463699f96bf2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:48.453023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c5cb8e6b-7086-45ad-bdd2-463699f96bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:48.562057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb8e6b-7086-45ad-bdd2-463699f96bf2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:48.574983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:48.637130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518caa6a-3679-4b01-a578-fc88265ed6a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:48.637770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518caa6a-3679-4b01-a578-fc88265ed6a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:48.638053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518caa6a-3679-4b01-a578-fc88265ed6a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:48.638327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518caa6a-3679-4b01-a578-fc88265ed6a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:48.639971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-518caa6a-3679-4b01-a578-fc88265ed6a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:48.754726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518caa6a-3679-4b01-a578-fc88265ed6a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:48.756904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518caa6a-3679-4b01-a578-fc88265ed6a9 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:48.776022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 651/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:50.774064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c24df-90a7-40ec-a314-59c596e02619 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:50.774630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c24df-90a7-40ec-a314-59c596e02619 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:50.774630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c24df-90a7-40ec-a314-59c596e02619 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:50.775829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 675/1326] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:50.780889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5668b2d-6683-416e-9d8a-8b987eba9eeb None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:50.781313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5668b2d-6683-416e-9d8a-8b987eba9eeb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:50.781441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5668b2d-6683-416e-9d8a-8b987eba9eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:50.831236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:50.849134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b5668b2d-6683-416e-9d8a-8b987eba9eeb None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:50.860268 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b5668b2d-6683-416e-9d8a-8b987eba9eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:50.951501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 652/1327] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:51.234459 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c24df-90a7-40ec-a314-59c596e02619 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:51.235990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-983c24df-90a7-40ec-a314-59c596e02619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:51.340263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c24df-90a7-40ec-a314-59c596e02619 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:51.424846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc5710-c97d-4296-bf04-ac490322e781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:51.425495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc5710-c97d-4296-bf04-ac490322e781 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:51.425760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc5710-c97d-4296-bf04-ac490322e781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:51.426028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc5710-c97d-4296-bf04-ac490322e781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:51.427739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a8dc5710-c97d-4296-bf04-ac490322e781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:51.444441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:51.537682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc5710-c97d-4296-bf04-ac490322e781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:51.539116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc5710-c97d-4296-bf04-ac490322e781 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:51.557749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 676/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:53.554056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df220-16cb-4441-adc8-9e50a67886f6 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:53.554538 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df220-16cb-4441-adc8-9e50a67886f6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:53.554648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df220-16cb-4441-adc8-9e50a67886f6 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:53.556037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 653/1329] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:22:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:53.561419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba92c4-2a6d-4a3a-9ea1-e2fabff88f9c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:53.561861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba92c4-2a6d-4a3a-9ea1-e2fabff88f9c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:53.562056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba92c4-2a6d-4a3a-9ea1-e2fabff88f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:53.636578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3eba92c4-2a6d-4a3a-9ea1-e2fabff88f9c None None] MFA Rules not processed for user `57a182ce8b7e4cadb8c50cc2a7fa67ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:53.637844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3eba92c4-2a6d-4a3a-9ea1-e2fabff88f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:53.727401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:53.758686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 677/1330] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:54.054710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2df220-16cb-4441-adc8-9e50a67886f6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:54.056472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-aa2df220-16cb-4441-adc8-9e50a67886f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:54.167261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2df220-16cb-4441-adc8-9e50a67886f6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:54.186927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:54.241577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55617522-8c5c-48eb-9360-227398835213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:54.242191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55617522-8c5c-48eb-9360-227398835213 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:54.242470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55617522-8c5c-48eb-9360-227398835213 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:54.242742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55617522-8c5c-48eb-9360-227398835213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:54.244355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-55617522-8c5c-48eb-9360-227398835213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:54.358367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55617522-8c5c-48eb-9360-227398835213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:54.359751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55617522-8c5c-48eb-9360-227398835213 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:54.377902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 654/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:22:56.755310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedfb64-97a3-4db3-a87f-da3dd171bfcd None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:56.756400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedfb64-97a3-4db3-a87f-da3dd171bfcd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:56.756682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedfb64-97a3-4db3-a87f-da3dd171bfcd None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:56.758134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 678/1332] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:56.763705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dc048-e0e7-45a0-981c-9771647d9b64 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:56.766786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dc048-e0e7-45a0-981c-9771647d9b64 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:56.766786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dc048-e0e7-45a0-981c-9771647d9b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:56.830901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:56.834226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-787dc048-e0e7-45a0-981c-9771647d9b64 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:56.835446 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-787dc048-e0e7-45a0-981c-9771647d9b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:56.932194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 655/1333] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:57.213736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedfb64-97a3-4db3-a87f-da3dd171bfcd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:57.215296 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5fedfb64-97a3-4db3-a87f-da3dd171bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:57.320603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedfb64-97a3-4db3-a87f-da3dd171bfcd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:57.398189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143386fb-b04f-44eb-9eb1-79655a3083e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:57.398804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143386fb-b04f-44eb-9eb1-79655a3083e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:57.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143386fb-b04f-44eb-9eb1-79655a3083e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:57.399346 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143386fb-b04f-44eb-9eb1-79655a3083e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:57.400949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-143386fb-b04f-44eb-9eb1-79655a3083e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:57.428281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:57.512882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143386fb-b04f-44eb-9eb1-79655a3083e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:57.514255 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143386fb-b04f-44eb-9eb1-79655a3083e3 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:57.532720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 679/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:59.501951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f60c97-794f-4392-a817-90f649474c97 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:59.502578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f60c97-794f-4392-a817-90f649474c97 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:59.502578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f60c97-794f-4392-a817-90f649474c97 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:59.503852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 656/1335] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:59.509580 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd511506-0a20-4800-83bc-2e5921192a42 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:59.510003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd511506-0a20-4800-83bc-2e5921192a42 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:59.510136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd511506-0a20-4800-83bc-2e5921192a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:59.575845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cd511506-0a20-4800-83bc-2e5921192a42 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:59.576981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cd511506-0a20-4800-83bc-2e5921192a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:59.650940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:59.674102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 680/1336] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:59.962687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f60c97-794f-4392-a817-90f649474c97 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:59.964216 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b7f60c97-794f-4392-a817-90f649474c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:00.067587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f60c97-794f-4392-a817-90f649474c97 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:00.095565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:00.141613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd5952-aff3-4992-a133-5c41a0a1c762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:00.142215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd5952-aff3-4992-a133-5c41a0a1c762 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:00.142489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd5952-aff3-4992-a133-5c41a0a1c762 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:00.142832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd5952-aff3-4992-a133-5c41a0a1c762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:00.144340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0fdd5952-aff3-4992-a133-5c41a0a1c762 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:00.251452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd5952-aff3-4992-a133-5c41a0a1c762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:00.252830 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd5952-aff3-4992-a133-5c41a0a1c762 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:00.270905 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 657/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:02.928963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d425d4-db76-4685-81b9-2e18758b207d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:02.929520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d425d4-db76-4685-81b9-2e18758b207d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:02.929520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d425d4-db76-4685-81b9-2e18758b207d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:02.930864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 681/1338] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:23:02.935973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be58816-67fd-4d99-aeda-db18f486431c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:02.936399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be58816-67fd-4d99-aeda-db18f486431c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:02.936561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be58816-67fd-4d99-aeda-db18f486431c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:03.003052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2be58816-67fd-4d99-aeda-db18f486431c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:03.004181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2be58816-67fd-4d99-aeda-db18f486431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:03.012664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:03.100893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 658/1339] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:03.386271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d425d4-db76-4685-81b9-2e18758b207d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:03.387834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a1d425d4-db76-4685-81b9-2e18758b207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:03.490130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d425d4-db76-4685-81b9-2e18758b207d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:03.561564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e9bc5-d048-4d99-8d19-3afda3e4eb26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:03.562096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9bc5-d048-4d99-8d19-3afda3e4eb26 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:03.562370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9bc5-d048-4d99-8d19-3afda3e4eb26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:03.562634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9bc5-d048-4d99-8d19-3afda3e4eb26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:03.564202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a92e9bc5-d048-4d99-8d19-3afda3e4eb26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:03.598971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:03.671595 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e9bc5-d048-4d99-8d19-3afda3e4eb26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:03.673022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e9bc5-d048-4d99-8d19-3afda3e4eb26 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:03.691630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 682/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:07.777922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517842b5-58b4-46c5-b46e-676fb813bef1 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:07.778495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517842b5-58b4-46c5-b46e-676fb813bef1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:07.778495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517842b5-58b4-46c5-b46e-676fb813bef1 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:07.780035 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 659/1341] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:07.785250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c7058-4a9e-44a3-9116-86cd0e599821 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:07.785715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c7058-4a9e-44a3-9116-86cd0e599821 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:07.796557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c7058-4a9e-44a3-9116-86cd0e599821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:07.854763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-724c7058-4a9e-44a3-9116-86cd0e599821 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:07.855967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-724c7058-4a9e-44a3-9116-86cd0e599821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:07.945979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:07.957908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 683/1342] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:08.242622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517842b5-58b4-46c5-b46e-676fb813bef1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:08.244140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-517842b5-58b4-46c5-b46e-676fb813bef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:08.349124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517842b5-58b4-46c5-b46e-676fb813bef1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:08.388143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:08.425774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358306d-3542-4f19-85d5-77b53d9704ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:08.426398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358306d-3542-4f19-85d5-77b53d9704ec service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:08.426757 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358306d-3542-4f19-85d5-77b53d9704ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:08.427042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358306d-3542-4f19-85d5-77b53d9704ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:08.428664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f358306d-3542-4f19-85d5-77b53d9704ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:08.545065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358306d-3542-4f19-85d5-77b53d9704ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:08.546454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358306d-3542-4f19-85d5-77b53d9704ec service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:08.565410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 660/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:11.499548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14faf3-fd3b-4ed5-a001-e2177cb6492c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:11.500134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14faf3-fd3b-4ed5-a001-e2177cb6492c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:11.500134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14faf3-fd3b-4ed5-a001-e2177cb6492c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:11.501437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 684/1344] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:11.507196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbeb12-06cd-4a4c-8c80-87bfcfae0fea None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:11.507618 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbeb12-06cd-4a4c-8c80-87bfcfae0fea None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:11.507720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbeb12-06cd-4a4c-8c80-87bfcfae0fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:11.575071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3edbeb12-06cd-4a4c-8c80-87bfcfae0fea None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:11.576226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3edbeb12-06cd-4a4c-8c80-87bfcfae0fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:11.595976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:11.681435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 661/1345] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:11.979781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14faf3-fd3b-4ed5-a001-e2177cb6492c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:11.981516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-da14faf3-fd3b-4ed5-a001-e2177cb6492c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:12.092765 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14faf3-fd3b-4ed5-a001-e2177cb6492c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:12.166377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c773b1-8f58-43c2-b7e6-55733c4baa1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:12.166986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c773b1-8f58-43c2-b7e6-55733c4baa1a service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:12.167264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c773b1-8f58-43c2-b7e6-55733c4baa1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:12.167564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c773b1-8f58-43c2-b7e6-55733c4baa1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:12.169211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-01c773b1-8f58-43c2-b7e6-55733c4baa1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:12.216927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:12.286745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c773b1-8f58-43c2-b7e6-55733c4baa1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:12.288205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c773b1-8f58-43c2-b7e6-55733c4baa1a service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:12.308108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 685/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:16.514755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8c30f-db36-4a6c-8785-2c383a5ff816 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:16.515367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8c30f-db36-4a6c-8785-2c383a5ff816 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:16.515367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8c30f-db36-4a6c-8785-2c383a5ff816 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:16.516587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 662/1347] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:16.522385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c056c-266d-4acd-8c32-6001607a8306 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:16.522899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c056c-266d-4acd-8c32-6001607a8306 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:16.522995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c056c-266d-4acd-8c32-6001607a8306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:16.595299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f90c056c-266d-4acd-8c32-6001607a8306 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:16.596561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f90c056c-266d-4acd-8c32-6001607a8306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:16.710458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:16.713723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 686/1348] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:23:16.999235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8c30f-db36-4a6c-8785-2c383a5ff816 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:17.000761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0ed8c30f-db36-4a6c-8785-2c383a5ff816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:17.107861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8c30f-db36-4a6c-8785-2c383a5ff816 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:17.155989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:17.181893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eed844-0905-47e1-bcbc-e3608f0a6a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:17.182504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eed844-0905-47e1-bcbc-e3608f0a6a2c service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:17.182783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eed844-0905-47e1-bcbc-e3608f0a6a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:17.183053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eed844-0905-47e1-bcbc-e3608f0a6a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:17.184902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-27eed844-0905-47e1-bcbc-e3608f0a6a2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:17.295759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eed844-0905-47e1-bcbc-e3608f0a6a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:17.297347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eed844-0905-47e1-bcbc-e3608f0a6a2c service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:17.321711 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 663/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:22.213502 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831e65a-18e7-4ae3-854a-ebc1af13b19c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:22.214475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831e65a-18e7-4ae3-854a-ebc1af13b19c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:22.214774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831e65a-18e7-4ae3-854a-ebc1af13b19c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:22.216436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 687/1350] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:22.222213 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031123b-f536-4b1d-b902-74ffafe24e1b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:22.222778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031123b-f536-4b1d-b902-74ffafe24e1b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:22.224361 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031123b-f536-4b1d-b902-74ffafe24e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:22.393096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f031123b-f536-4b1d-b902-74ffafe24e1b None None] MFA Rules not processed for user `c11c7fb612ab40d88f526b8b38dbfb23`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:22.394762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f031123b-f536-4b1d-b902-74ffafe24e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:22.419087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:22.528370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 664/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:23:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:26.656661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db57dc-dbb2-40a0-ae9f-ef595b0ced05 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:26.657578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db57dc-dbb2-40a0-ae9f-ef595b0ced05 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:26.658021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db57dc-dbb2-40a0-ae9f-ef595b0ced05 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:26.659528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 688/1352] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:26.665041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc3a56-540a-49ea-b861-fb4cc1405185 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:26.665514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc3a56-540a-49ea-b861-fb4cc1405185 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:26.665621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc3a56-540a-49ea-b861-fb4cc1405185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:26.756661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-68cc3a56-540a-49ea-b861-fb4cc1405185 None None] MFA Rules not processed for user `c11c7fb612ab40d88f526b8b38dbfb23`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:26.758233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-68cc3a56-540a-49ea-b861-fb4cc1405185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:26.840716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:26.877823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 665/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:23:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:28.318787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be990e1-a5b4-47e5-9d0e-1f98a329b895 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:28.319317 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be990e1-a5b4-47e5-9d0e-1f98a329b895 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:28.319317 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be990e1-a5b4-47e5-9d0e-1f98a329b895 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:28.320898 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 689/1354] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:28.325888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a29f9e-3121-41ef-a6e6-f82886e1688a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:28.326312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a29f9e-3121-41ef-a6e6-f82886e1688a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:28.326430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a29f9e-3121-41ef-a6e6-f82886e1688a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:28.405494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b0a29f9e-3121-41ef-a6e6-f82886e1688a None None] MFA Rules not processed for user `c11c7fb612ab40d88f526b8b38dbfb23`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:28.406695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b0a29f9e-3121-41ef-a6e6-f82886e1688a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:28.516997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 666/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:23:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:23:31.909581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728774c2-d42e-4749-b0c9-ef5f6261d6a3 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:31.910216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728774c2-d42e-4749-b0c9-ef5f6261d6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:31.910499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728774c2-d42e-4749-b0c9-ef5f6261d6a3 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:31.912319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 690/1356] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:31.917661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e5bf5-a7f5-4f2f-91e0-215802a368d3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:31.918106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e5bf5-a7f5-4f2f-91e0-215802a368d3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:31.918188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e5bf5-a7f5-4f2f-91e0-215802a368d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:31.969151 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:32.008246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e88e5bf5-a7f5-4f2f-91e0-215802a368d3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:32.009580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e88e5bf5-a7f5-4f2f-91e0-215802a368d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:32.142732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 667/1357] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:32.443846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80453a78-1ce5-488a-94c5-5816a525d290 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:32.444778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80453a78-1ce5-488a-94c5-5816a525d290 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:32.445719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80453a78-1ce5-488a-94c5-5816a525d290 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:32.448395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 691/1358] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 1 05:23:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:32.455781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894f11a-c9e4-4587-8807-89447d5f219d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:32.456062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894f11a-c9e4-4587-8807-89447d5f219d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:32.456347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894f11a-c9e4-4587-8807-89447d5f219d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:32.550684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0894f11a-c9e4-4587-8807-89447d5f219d None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:32.552549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0894f11a-c9e4-4587-8807-89447d5f219d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:32.612499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:32.683294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 668/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 1 05:23:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:23:32.691340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80453a78-1ce5-488a-94c5-5816a525d290 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:32.692851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-80453a78-1ce5-488a-94c5-5816a525d290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:32.863236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80453a78-1ce5-488a-94c5-5816a525d290 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:32.942764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:32.984633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e35f27-66af-426d-9d96-392232e7a96e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:32.985761 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e35f27-66af-426d-9d96-392232e7a96e service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:32.985761 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e35f27-66af-426d-9d96-392232e7a96e service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:32.986182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e35f27-66af-426d-9d96-392232e7a96e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:32.988572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-34e35f27-66af-426d-9d96-392232e7a96e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:33.082674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e35f27-66af-426d-9d96-392232e7a96e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:33.083682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e35f27-66af-426d-9d96-392232e7a96e service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:33.103016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 692/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:35.252469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20cde1-1450-476d-b578-52234e4bf133 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:35.253159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20cde1-1450-476d-b578-52234e4bf133 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:35.253159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20cde1-1450-476d-b578-52234e4bf133 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:35.255001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 669/1361] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:35.260176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9321d-c6ba-4ba8-bd54-568ef63112a4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:35.260699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9321d-c6ba-4ba8-bd54-568ef63112a4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:35.260699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9321d-c6ba-4ba8-bd54-568ef63112a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:35.330861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4cc9321d-c6ba-4ba8-bd54-568ef63112a4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:35.332412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4cc9321d-c6ba-4ba8-bd54-568ef63112a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:35.373384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:35.434798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 693/1362] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:35.736818 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20cde1-1450-476d-b578-52234e4bf133 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:35.736818 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8e20cde1-1450-476d-b578-52234e4bf133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:35.899401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20cde1-1450-476d-b578-52234e4bf133 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:35.909413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:36.021574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf4ebd-7c04-47c4-833b-6db5ebc7374f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:36.022181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4ebd-7c04-47c4-833b-6db5ebc7374f service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:36.022453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4ebd-7c04-47c4-833b-6db5ebc7374f service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:36.022714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4ebd-7c04-47c4-833b-6db5ebc7374f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:36.024301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-48cf4ebd-7c04-47c4-833b-6db5ebc7374f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:36.121419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf4ebd-7c04-47c4-833b-6db5ebc7374f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:36.123053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf4ebd-7c04-47c4-833b-6db5ebc7374f service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:36.142951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 670/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:38.582668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b62e72-836d-4b5a-9a55-c7b769f74252 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:38.583546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b62e72-836d-4b5a-9a55-c7b769f74252 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:38.583818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b62e72-836d-4b5a-9a55-c7b769f74252 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:38.585356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 694/1364] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:38.590808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfa2ed-d3c1-45e3-bfae-3e6ae1106e1c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:38.591231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfa2ed-d3c1-45e3-bfae-3e6ae1106e1c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:38.591333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfa2ed-d3c1-45e3-bfae-3e6ae1106e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:38.608307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:38.661611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4fcfa2ed-d3c1-45e3-bfae-3e6ae1106e1c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:38.663474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4fcfa2ed-d3c1-45e3-bfae-3e6ae1106e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:38.782800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 671/1365] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:39.094515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857ac11-d0ba-479e-a490-3f05481fe0b0 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:39.094820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857ac11-d0ba-479e-a490-3f05481fe0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:39.095201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857ac11-d0ba-479e-a490-3f05481fe0b0 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:39.096640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 695/1366] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 1 05:23:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:39.102839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523386c-471c-41df-a839-cbf2e195e40f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:39.103964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523386c-471c-41df-a839-cbf2e195e40f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:39.103964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523386c-471c-41df-a839-cbf2e195e40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:39.192820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0523386c-471c-41df-a839-cbf2e195e40f None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:39.194080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0523386c-471c-41df-a839-cbf2e195e40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:39.237174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:39.360629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 672/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 1 05:23:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:39.360999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857ac11-d0ba-479e-a490-3f05481fe0b0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:39.360999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2857ac11-d0ba-479e-a490-3f05481fe0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:39.522204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857ac11-d0ba-479e-a490-3f05481fe0b0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:39.545814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:39.655268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956efb29-d4c8-4fcb-8371-b6b8166153e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:39.655976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956efb29-d4c8-4fcb-8371-b6b8166153e7 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:39.656074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956efb29-d4c8-4fcb-8371-b6b8166153e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:39.656337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956efb29-d4c8-4fcb-8371-b6b8166153e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:39.658018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-956efb29-d4c8-4fcb-8371-b6b8166153e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:39.763697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956efb29-d4c8-4fcb-8371-b6b8166153e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:39.765269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956efb29-d4c8-4fcb-8371-b6b8166153e7 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:39.784258 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 696/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:23:40.277434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccaf21-14b2-410e-8a6e-dccf0ca9d505 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:40.278006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccaf21-14b2-410e-8a6e-dccf0ca9d505 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:40.278006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccaf21-14b2-410e-8a6e-dccf0ca9d505 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:40.279456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 673/1369] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:40.284586 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b6b6-8ff7-4149-9e09-25eba0ef6e50 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:40.284984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b6b6-8ff7-4149-9e09-25eba0ef6e50 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:40.285266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b6b6-8ff7-4149-9e09-25eba0ef6e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:40.317880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:40.354938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ce16b6b6-8ff7-4149-9e09-25eba0ef6e50 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:40.356291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ce16b6b6-8ff7-4149-9e09-25eba0ef6e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:40.459455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 697/1370] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:41.049378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27764-3e29-473e-a236-766d08a9c1b4 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:41.049707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27764-3e29-473e-a236-766d08a9c1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:41.050532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27764-3e29-473e-a236-766d08a9c1b4 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:41.052033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 674/1371] 10.222.0.42 () {54 vars in 964 bytes} [Thu Sep 1 05:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:41.057195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00311523-1f3d-4046-9c38-e5f8cb5d9871 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:41.057642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00311523-1f3d-4046-9c38-e5f8cb5d9871 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:41.057840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00311523-1f3d-4046-9c38-e5f8cb5d9871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:41.139421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-00311523-1f3d-4046-9c38-e5f8cb5d9871 None None] MFA Rules not processed for user `7e0e8589931b46ca84168a2ad273b63d`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:41.140637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-00311523-1f3d-4046-9c38-e5f8cb5d9871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:41.197821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:41.226263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 698/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Sep 1 05:23:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:41.233885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f27764-3e29-473e-a236-766d08a9c1b4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:41.235374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-32f27764-3e29-473e-a236-766d08a9c1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:41.357550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f27764-3e29-473e-a236-766d08a9c1b4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:41.399525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:41.438807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411382e-6dff-4b52-9bc8-9faa821d9f79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:41.439335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411382e-6dff-4b52-9bc8-9faa821d9f79 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:41.439660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411382e-6dff-4b52-9bc8-9faa821d9f79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:41.440002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411382e-6dff-4b52-9bc8-9faa821d9f79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:41.443098 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1411382e-6dff-4b52-9bc8-9faa821d9f79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:41.574577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411382e-6dff-4b52-9bc8-9faa821d9f79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:41.575613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411382e-6dff-4b52-9bc8-9faa821d9f79 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:41.598581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 675/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:42.008888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4d88-5602-437f-a50f-d54d2c54f3a7 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:42.009180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4d88-5602-437f-a50f-d54d2c54f3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:42.009486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4d88-5602-437f-a50f-d54d2c54f3a7 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:42.010844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 699/1374] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:42.016198 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812576a2-5d7b-403e-9949-7c92a5cec3da None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:42.016489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812576a2-5d7b-403e-9949-7c92a5cec3da None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:42.016771 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812576a2-5d7b-403e-9949-7c92a5cec3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:42.083263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-812576a2-5d7b-403e-9949-7c92a5cec3da None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:42.084487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-812576a2-5d7b-403e-9949-7c92a5cec3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:42.105133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:42.183365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 676/1375] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:42.478482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d4d88-5602-437f-a50f-d54d2c54f3a7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:42.480004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d70d4d88-5602-437f-a50f-d54d2c54f3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:42.642136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d4d88-5602-437f-a50f-d54d2c54f3a7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:42.695365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:42.759450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a65aa-6b67-475c-9b49-f9cea77fda2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:42.760188 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a65aa-6b67-475c-9b49-f9cea77fda2f service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:42.760614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a65aa-6b67-475c-9b49-f9cea77fda2f service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:42.760891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a65aa-6b67-475c-9b49-f9cea77fda2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:42.762565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a30a65aa-6b67-475c-9b49-f9cea77fda2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:42.854424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a65aa-6b67-475c-9b49-f9cea77fda2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:42.856018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a65aa-6b67-475c-9b49-f9cea77fda2f service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:42.874320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 700/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:45.176875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5489b-bb45-42c6-bd00-a71f5411f190 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:45.177339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5489b-bb45-42c6-bd00-a71f5411f190 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:45.177497 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5489b-bb45-42c6-bd00-a71f5411f190 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:45.178777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 677/1377] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:45.183887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74889eff-7cc5-4a99-8480-dab2a1fbc512 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:45.184282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74889eff-7cc5-4a99-8480-dab2a1fbc512 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:45.184470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74889eff-7cc5-4a99-8480-dab2a1fbc512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:45.258388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-74889eff-7cc5-4a99-8480-dab2a1fbc512 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:45.259600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-74889eff-7cc5-4a99-8480-dab2a1fbc512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:45.266702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:45.386113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 701/1378] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:45.691078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5489b-bb45-42c6-bd00-a71f5411f190 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:45.692565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-49a5489b-bb45-42c6-bd00-a71f5411f190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:45.854968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5489b-bb45-42c6-bd00-a71f5411f190 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:45.858988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:45.977134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f603039-56cc-47d4-896d-d9866dc085e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:45.977820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f603039-56cc-47d4-896d-d9866dc085e5 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:45.978100 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f603039-56cc-47d4-896d-d9866dc085e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:45.978370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f603039-56cc-47d4-896d-d9866dc085e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:45.979972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7f603039-56cc-47d4-896d-d9866dc085e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:46.077199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f603039-56cc-47d4-896d-d9866dc085e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:46.078857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f603039-56cc-47d4-896d-d9866dc085e5 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:46.098672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 678/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:48.286418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44ed21-17d7-4120-908f-5742f9e45cca None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:48.287050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44ed21-17d7-4120-908f-5742f9e45cca None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:48.287050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44ed21-17d7-4120-908f-5742f9e45cca None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:48.288581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 702/1380] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:48.294547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0559e9-41ec-4b05-a447-439d3f0f30b3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:48.295112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0559e9-41ec-4b05-a447-439d3f0f30b3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:48.295112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0559e9-41ec-4b05-a447-439d3f0f30b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:48.308080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:48.370291 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5d0559e9-41ec-4b05-a447-439d3f0f30b3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:48.371503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5d0559e9-41ec-4b05-a447-439d3f0f30b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:48.475490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 679/1381] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:48.774345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44ed21-17d7-4120-908f-5742f9e45cca None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:48.775914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1a44ed21-17d7-4120-908f-5742f9e45cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:48.925151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:48.935570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44ed21-17d7-4120-908f-5742f9e45cca None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:49.056888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6805a-5d6d-4008-8bb9-74c2ea3677a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:49.057413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6805a-5d6d-4008-8bb9-74c2ea3677a8 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:49.057724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6805a-5d6d-4008-8bb9-74c2ea3677a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:49.057999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6805a-5d6d-4008-8bb9-74c2ea3677a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:49.059601 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bca6805a-5d6d-4008-8bb9-74c2ea3677a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:49.153986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6805a-5d6d-4008-8bb9-74c2ea3677a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:49.155610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6805a-5d6d-4008-8bb9-74c2ea3677a8 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:49.165923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:49.174210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 703/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:51.321237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551859ab-18b5-4203-b28d-54779336dbd3 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:51.321806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551859ab-18b5-4203-b28d-54779336dbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:51.321970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551859ab-18b5-4203-b28d-54779336dbd3 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:51.323898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 680/1383] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:51.329137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba86261-33b5-4d9c-a617-aa8e6b4e5ead None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:51.329591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba86261-33b5-4d9c-a617-aa8e6b4e5ead None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:51.329751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba86261-33b5-4d9c-a617-aa8e6b4e5ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:51.401199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3ba86261-33b5-4d9c-a617-aa8e6b4e5ead None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:51.402556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3ba86261-33b5-4d9c-a617-aa8e6b4e5ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:51.504964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 704/1384] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:51.795239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551859ab-18b5-4203-b28d-54779336dbd3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:51.797086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-551859ab-18b5-4203-b28d-54779336dbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:51.899058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:51.969718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551859ab-18b5-4203-b28d-54779336dbd3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:52.091337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a2dbc-1713-4c54-b61e-0cbc607f5f13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:52.091947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2dbc-1713-4c54-b61e-0cbc607f5f13 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:52.092224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2dbc-1713-4c54-b61e-0cbc607f5f13 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:52.092531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a2dbc-1713-4c54-b61e-0cbc607f5f13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:52.094117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a09a2dbc-1713-4c54-b61e-0cbc607f5f13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:52.145428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:52.198907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a2dbc-1713-4c54-b61e-0cbc607f5f13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:52.200538 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a2dbc-1713-4c54-b61e-0cbc607f5f13 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:52.220001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 681/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:54.369743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fffa4c-7fbf-4092-80ea-46b59905be87 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:54.370313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fffa4c-7fbf-4092-80ea-46b59905be87 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:54.370313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fffa4c-7fbf-4092-80ea-46b59905be87 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:54.371790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 705/1386] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:54.377899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b38ed4-37fc-4a8d-9e6c-ad320976f959 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:54.378343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b38ed4-37fc-4a8d-9e6c-ad320976f959 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:54.378596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b38ed4-37fc-4a8d-9e6c-ad320976f959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:54.454945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-83b38ed4-37fc-4a8d-9e6c-ad320976f959 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:54.457278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-83b38ed4-37fc-4a8d-9e6c-ad320976f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:54.581494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 682/1387] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:54.893007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fffa4c-7fbf-4092-80ea-46b59905be87 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:54.894658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c0fffa4c-7fbf-4092-80ea-46b59905be87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:54.976533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:55.052369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fffa4c-7fbf-4092-80ea-46b59905be87 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:55.171959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09a12a-9031-412b-bf9b-e323fbd351ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:55.172571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a12a-9031-412b-bf9b-e323fbd351ca service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:55.172849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a12a-9031-412b-bf9b-e323fbd351ca service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:55.173119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a12a-9031-412b-bf9b-e323fbd351ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:55.175459 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3f09a12a-9031-412b-bf9b-e323fbd351ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:55.215953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:55.270065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09a12a-9031-412b-bf9b-e323fbd351ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:55.271690 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09a12a-9031-412b-bf9b-e323fbd351ca service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:55.290656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 706/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:57.112759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1a7de-aece-4e1f-a9d2-869cdc1710cf None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:57.113336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1a7de-aece-4e1f-a9d2-869cdc1710cf None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:57.113336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1a7de-aece-4e1f-a9d2-869cdc1710cf None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:57.114729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 683/1389] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:57.120125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27d480-9feb-4fc0-b53b-0097ef8824e1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:57.120528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27d480-9feb-4fc0-b53b-0097ef8824e1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:57.120671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27d480-9feb-4fc0-b53b-0097ef8824e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:57.186447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4b27d480-9feb-4fc0-b53b-0097ef8824e1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:57.187640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4b27d480-9feb-4fc0-b53b-0097ef8824e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:57.287865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 707/1390] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:57.374869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801e394-f60d-4343-a7e0-28cca42b8621 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:57.375094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801e394-f60d-4343-a7e0-28cca42b8621 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:57.375387 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801e394-f60d-4343-a7e0-28cca42b8621 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:57.376760 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 684/1391] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:57.382581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521c81a-ce75-4ec3-ad32-f7dd4659eafd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:57.382872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521c81a-ce75-4ec3-ad32-f7dd4659eafd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:57.383151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521c81a-ce75-4ec3-ad32-f7dd4659eafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:57.398702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:57.452006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0521c81a-ce75-4ec3-ad32-f7dd4659eafd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:57.453208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0521c81a-ce75-4ec3-ad32-f7dd4659eafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:57.553859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 708/1392] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:57.592192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b87ec-f21f-44ed-8c5b-86b7fab81103 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:57.592544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b87ec-f21f-44ed-8c5b-86b7fab81103 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:57.592932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b87ec-f21f-44ed-8c5b-86b7fab81103 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:57.594377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 685/1393] 10.222.0.42 () {54 vars in 967 bytes} [Thu Sep 1 05:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:57.599691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dd89a-3faf-4075-b067-7916526b6f25 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:57.599979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dd89a-3faf-4075-b067-7916526b6f25 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:57.600260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dd89a-3faf-4075-b067-7916526b6f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:57.677946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-989dd89a-3faf-4075-b067-7916526b6f25 None None] MFA Rules not processed for user `4c23597e0da2454abd2a5354d779b092`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:57.679094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-989dd89a-3faf-4075-b067-7916526b6f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:57.715804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:57.760982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 709/1394] 10.222.0.42 () {58 vars in 1064 bytes} [Thu Sep 1 05:23:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:23:57.768647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b87ec-f21f-44ed-8c5b-86b7fab81103 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:57.770188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8f9b87ec-f21f-44ed-8c5b-86b7fab81103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:57.802374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:57.849615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dd89a-3faf-4075-b067-7916526b6f25 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:57.850822 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-989dd89a-3faf-4075-b067-7916526b6f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:57.883392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b87ec-f21f-44ed-8c5b-86b7fab81103 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:57.963092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ea1fb-a3e1-492b-8fd1-b2f2c7e233de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c23597e0da2454abd2a5354d779b092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:57.963625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ea1fb-a3e1-492b-8fd1-b2f2c7e233de service glance] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:57.963904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ea1fb-a3e1-492b-8fd1-b2f2c7e233de service glance] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:57.964179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ea1fb-a3e1-492b-8fd1-b2f2c7e233de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:57.965827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-df6ea1fb-a3e1-492b-8fd1-b2f2c7e233de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:57.979001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dd89a-3faf-4075-b067-7916526b6f25 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:58.044991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:58.047502 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:58.082202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ea1fb-a3e1-492b-8fd1-b2f2c7e233de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:58.083679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ea1fb-a3e1-492b-8fd1-b2f2c7e233de service glance] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:58.098027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa9bba-c3ed-4f94-bec0-763b55fb103a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:58.098649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa9bba-c3ed-4f94-bec0-763b55fb103a service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:58.098933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa9bba-c3ed-4f94-bec0-763b55fb103a service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:58.099206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa9bba-c3ed-4f94-bec0-763b55fb103a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:58.100820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-04aa9bba-c3ed-4f94-bec0-763b55fb103a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:58.102215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 686/1395] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:58.195653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa9bba-c3ed-4f94-bec0-763b55fb103a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:58.197257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa9bba-c3ed-4f94-bec0-763b55fb103a service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:58.224409 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 710/1396] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:23:59.046684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5db23-e8a0-4cd0-a397-190042f89f85 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:59.047161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5db23-e8a0-4cd0-a397-190042f89f85 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:59.047396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5db23-e8a0-4cd0-a397-190042f89f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:59.117723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2ab5db23-e8a0-4cd0-a397-190042f89f85 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:59.118941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2ab5db23-e8a0-4cd0-a397-190042f89f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:59.206810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 687/1397] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:23:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:59.248480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74067bc-a720-4836-bafe-186b9dde6c81 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:59.248963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74067bc-a720-4836-bafe-186b9dde6c81 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:59.249059 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74067bc-a720-4836-bafe-186b9dde6c81 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:59.250419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 711/1398] 10.222.0.42 () {54 vars in 943 bytes} [Thu Sep 1 05:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:59.257478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4ebc1-fd5d-4fce-89e5-02db30093801 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:59.257766 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4ebc1-fd5d-4fce-89e5-02db30093801 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:59.258042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4ebc1-fd5d-4fce-89e5-02db30093801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:59.315641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:59.338181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-43e4ebc1-fd5d-4fce-89e5-02db30093801 None None] MFA Rules not processed for user `b723681faf15434e8529e8fc56c6e9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:59.339533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-43e4ebc1-fd5d-4fce-89e5-02db30093801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:59.448235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 688/1399] 10.222.0.42 () {58 vars in 1040 bytes} [Thu Sep 1 05:23:59 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:59.458597 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74067bc-a720-4836-bafe-186b9dde6c81 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:59.460066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f74067bc-a720-4836-bafe-186b9dde6c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:59.543242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:59.606062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74067bc-a720-4836-bafe-186b9dde6c81 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:59.710843 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddb6c5-adbe-4da0-9d5a-4d0e570f8207 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:59.711795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddb6c5-adbe-4da0-9d5a-4d0e570f8207 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:59.712072 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddb6c5-adbe-4da0-9d5a-4d0e570f8207 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:59.712343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddb6c5-adbe-4da0-9d5a-4d0e570f8207 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:59.714017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e1ddb6c5-adbe-4da0-9d5a-4d0e570f8207 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:59.778767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:59.802465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddb6c5-adbe-4da0-9d5a-4d0e570f8207 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:59.804268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddb6c5-adbe-4da0-9d5a-4d0e570f8207 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:59.812501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 712/1400] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:23:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:24:00.353931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b8d65-36ff-44ca-8a27-fe8f0844a8e8 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:00.354711 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b8d65-36ff-44ca-8a27-fe8f0844a8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:00.354988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b8d65-36ff-44ca-8a27-fe8f0844a8e8 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:00.356718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 689/1401] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:24:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:00.362272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68874544-b319-4ff9-9be4-9a9c4155af3a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:00.363437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68874544-b319-4ff9-9be4-9a9c4155af3a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:00.364073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68874544-b319-4ff9-9be4-9a9c4155af3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:00.442550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-68874544-b319-4ff9-9be4-9a9c4155af3a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:00.443770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-68874544-b319-4ff9-9be4-9a9c4155af3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:00.550692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 713/1402] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:00.847721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b8d65-36ff-44ca-8a27-fe8f0844a8e8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:00.849321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-953b8d65-36ff-44ca-8a27-fe8f0844a8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:00.964624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:01.017046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b8d65-36ff-44ca-8a27-fe8f0844a8e8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:01.148432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01825c62-08c8-44cf-8e3f-b1585d47ca81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:01.149229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01825c62-08c8-44cf-8e3f-b1585d47ca81 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:01.149609 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01825c62-08c8-44cf-8e3f-b1585d47ca81 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:01.149903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01825c62-08c8-44cf-8e3f-b1585d47ca81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:01.151468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-01825c62-08c8-44cf-8e3f-b1585d47ca81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:01.234073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:01.257161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01825c62-08c8-44cf-8e3f-b1585d47ca81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:01.259304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01825c62-08c8-44cf-8e3f-b1585d47ca81 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:01.279480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 690/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:24:03.376083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f34b20-80ae-4b8c-bcda-ef6ed6be0c7d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:03.376661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f34b20-80ae-4b8c-bcda-ef6ed6be0c7d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:03.376661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f34b20-80ae-4b8c-bcda-ef6ed6be0c7d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:03.378729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 714/1404] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:24:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:03.385689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932720c5-e729-470b-b42e-6597814eafd2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:03.386174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932720c5-e729-470b-b42e-6597814eafd2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:03.386393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932720c5-e729-470b-b42e-6597814eafd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:03.464076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-932720c5-e729-470b-b42e-6597814eafd2 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:03.465291 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-932720c5-e729-470b-b42e-6597814eafd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:03.579494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 691/1405] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:03.875029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f34b20-80ae-4b8c-bcda-ef6ed6be0c7d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:03.876574 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-89f34b20-80ae-4b8c-bcda-ef6ed6be0c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:03.951657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:04.063823 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f34b20-80ae-4b8c-bcda-ef6ed6be0c7d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:04.200936 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fabd9-c33a-42e3-a847-25ab5bbab948 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:04.201501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fabd9-c33a-42e3-a847-25ab5bbab948 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:04.201895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fabd9-c33a-42e3-a847-25ab5bbab948 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:04.202176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fabd9-c33a-42e3-a847-25ab5bbab948 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:04.203786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b98fabd9-c33a-42e3-a847-25ab5bbab948 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:04.218698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:04.308568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fabd9-c33a-42e3-a847-25ab5bbab948 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:04.310200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fabd9-c33a-42e3-a847-25ab5bbab948 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:04.330920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 715/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:06.423280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e586b-2d62-4671-806f-77608bd2635c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:06.424080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e586b-2d62-4671-806f-77608bd2635c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:06.424080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e586b-2d62-4671-806f-77608bd2635c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:06.425600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 692/1407] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:06.430902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906098e-3d3e-4b91-bfb0-7b73a3de62b1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:06.431449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906098e-3d3e-4b91-bfb0-7b73a3de62b1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:06.431449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906098e-3d3e-4b91-bfb0-7b73a3de62b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:06.501288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5906098e-3d3e-4b91-bfb0-7b73a3de62b1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:06.502608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5906098e-3d3e-4b91-bfb0-7b73a3de62b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:06.596899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:06.610581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 716/1408] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:24:06.902699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e586b-2d62-4671-806f-77608bd2635c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:06.904318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a35e586b-2d62-4671-806f-77608bd2635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:06.962913 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:07.077013 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e586b-2d62-4671-806f-77608bd2635c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:07.210386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a19a5f-85e5-4488-9c3d-cbd98c1c65ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:07.211003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a19a5f-85e5-4488-9c3d-cbd98c1c65ca service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:07.211281 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a19a5f-85e5-4488-9c3d-cbd98c1c65ca service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:07.211552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a19a5f-85e5-4488-9c3d-cbd98c1c65ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:07.213303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-13a19a5f-85e5-4488-9c3d-cbd98c1c65ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:07.221388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:07.316607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a19a5f-85e5-4488-9c3d-cbd98c1c65ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:07.318288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a19a5f-85e5-4488-9c3d-cbd98c1c65ca service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:07.339390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 693/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:16.645340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:17.335481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=42 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:23.068653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcab0a-99e6-4ec2-88fb-07382b1e6863 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:23.069206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcab0a-99e6-4ec2-88fb-07382b1e6863 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:23.069206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcab0a-99e6-4ec2-88fb-07382b1e6863 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:23.070811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 717/1410] 10.222.0.42 () {54 vars in 918 bytes} [Thu Sep 1 05:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:23.076052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170984d-d3e1-4e51-a101-bfd8672182b2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:23.076461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170984d-d3e1-4e51-a101-bfd8672182b2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:23.076594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170984d-d3e1-4e51-a101-bfd8672182b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:23.154131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0170984d-d3e1-4e51-a101-bfd8672182b2 None None] MFA Rules not processed for user `c11c7fb612ab40d88f526b8b38dbfb23`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:23.155336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0170984d-d3e1-4e51-a101-bfd8672182b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:23.261096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 694/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Sep 1 05:24:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:23.275858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0e75f-2acb-424d-99b0-98a94c893651 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:23.276143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0e75f-2acb-424d-99b0-98a94c893651 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:23.276414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0e75f-2acb-424d-99b0-98a94c893651 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:23.277779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 718/1412] 10.222.0.42 () {54 vars in 918 bytes} [Thu Sep 1 05:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:23.282192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90288b0-ef98-411d-b48b-a2851b93a450 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:23.282494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90288b0-ef98-411d-b48b-a2851b93a450 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:23.282751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90288b0-ef98-411d-b48b-a2851b93a450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:23.356560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:23.359258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c90288b0-ef98-411d-b48b-a2851b93a450 None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:23.360708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c90288b0-ef98-411d-b48b-a2851b93a450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:23.479672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 695/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Sep 1 05:24:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:25.867636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945fce5-135c-4552-bfe0-856ea8b0202b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:25.868190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945fce5-135c-4552-bfe0-856ea8b0202b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:25.868190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945fce5-135c-4552-bfe0-856ea8b0202b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:25.869528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:25.874839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249e9f7-17a9-490a-b041-c30e6970055b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:25.875222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249e9f7-17a9-490a-b041-c30e6970055b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:25.875398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249e9f7-17a9-490a-b041-c30e6970055b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:25.943039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4249e9f7-17a9-490a-b041-c30e6970055b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:25.944235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4249e9f7-17a9-490a-b041-c30e6970055b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:25.997896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:26.040818 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:26.328458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945fce5-135c-4552-bfe0-856ea8b0202b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:26.330691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1945fce5-135c-4552-bfe0-856ea8b0202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:26.487376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945fce5-135c-4552-bfe0-856ea8b0202b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:26.558893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:26.600307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6327f8-4b36-4233-8f23-2155abb6fead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:26.600914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6327f8-4b36-4233-8f23-2155abb6fead service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:26.601191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6327f8-4b36-4233-8f23-2155abb6fead service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:26.601482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6327f8-4b36-4233-8f23-2155abb6fead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:26.603157 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8f6327f8-4b36-4233-8f23-2155abb6fead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:26.691536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6327f8-4b36-4233-8f23-2155abb6fead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:26.693120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6327f8-4b36-4233-8f23-2155abb6fead service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:26.710847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 720/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:27.622648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9de6a-884d-494b-a0c9-05d7deeffa45 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:27.623176 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9de6a-884d-494b-a0c9-05d7deeffa45 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:27.623176 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9de6a-884d-494b-a0c9-05d7deeffa45 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:27.624547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:27.629826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8621e-c064-42c6-9223-ed4572aa6b03 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:27.630245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8621e-c064-42c6-9223-ed4572aa6b03 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:27.630383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8621e-c064-42c6-9223-ed4572aa6b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:27.708769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-60d8621e-c064-42c6-9223-ed4572aa6b03 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:27.710183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-60d8621e-c064-42c6-9223-ed4572aa6b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:27.739559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:27.810405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:27.962890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9de6a-884d-494b-a0c9-05d7deeffa45 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:27.964317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f1e9de6a-884d-494b-a0c9-05d7deeffa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:28.114728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9de6a-884d-494b-a0c9-05d7deeffa45 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:28.156347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:28.230895 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e26d6-78ff-44f1-8487-e04207cb2848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:28.231413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e26d6-78ff-44f1-8487-e04207cb2848 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:28.231691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e26d6-78ff-44f1-8487-e04207cb2848 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:28.231956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e26d6-78ff-44f1-8487-e04207cb2848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:28.233569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-789e26d6-78ff-44f1-8487-e04207cb2848 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:28.322533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e26d6-78ff-44f1-8487-e04207cb2848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:28.324186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e26d6-78ff-44f1-8487-e04207cb2848 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:28.342647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 698/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:29.236105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00684c63-6671-41a2-8a44-46406c85c4c3 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:29.236764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00684c63-6671-41a2-8a44-46406c85c4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:29.236764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00684c63-6671-41a2-8a44-46406c85c4c3 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:29.238062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:24:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:29.242985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56771334-55ee-4cfb-b961-dd9105241ad4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:29.243432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56771334-55ee-4cfb-b961-dd9105241ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:29.243548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56771334-55ee-4cfb-b961-dd9105241ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:29.418372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:29.440649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-56771334-55ee-4cfb-b961-dd9105241ad4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:29.441967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-56771334-55ee-4cfb-b961-dd9105241ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:29.537762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:29.692268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00684c63-6671-41a2-8a44-46406c85c4c3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:29.693856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-00684c63-6671-41a2-8a44-46406c85c4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:29.843787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00684c63-6671-41a2-8a44-46406c85c4c3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:29.856772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:29.958152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ccd3d-9692-44de-be7b-173daf1b7ae0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:29.958760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ccd3d-9692-44de-be7b-173daf1b7ae0 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:29.959034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ccd3d-9692-44de-be7b-173daf1b7ae0 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:29.959304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ccd3d-9692-44de-be7b-173daf1b7ae0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:29.960911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4b7ccd3d-9692-44de-be7b-173daf1b7ae0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:30.054926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ccd3d-9692-44de-be7b-173daf1b7ae0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:30.056530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ccd3d-9692-44de-be7b-173daf1b7ae0 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:30.075044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 723/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:30.934014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb09c9a-d291-40f8-bde6-be42f2940304 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:30.934611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb09c9a-d291-40f8-bde6-be42f2940304 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:30.934824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb09c9a-d291-40f8-bde6-be42f2940304 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:30.936242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:24:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:30.941320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0b0b-7e5b-40dc-b045-44f403addc08 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:30.941874 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0b0b-7e5b-40dc-b045-44f403addc08 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:30.941987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0b0b-7e5b-40dc-b045-44f403addc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:30.962280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:31.018398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-acaf0b0b-7e5b-40dc-b045-44f403addc08 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:31.019530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-acaf0b0b-7e5b-40dc-b045-44f403addc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:31.116590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:31.267332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb09c9a-d291-40f8-bde6-be42f2940304 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:31.268870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fcb09c9a-d291-40f8-bde6-be42f2940304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:31.377955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:31.413565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb09c9a-d291-40f8-bde6-be42f2940304 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:31.525885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7bd04-bcc8-4941-81c2-36c7214168ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:31.526407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7bd04-bcc8-4941-81c2-36c7214168ca service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:31.526680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7bd04-bcc8-4941-81c2-36c7214168ca service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:31.526945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7bd04-bcc8-4941-81c2-36c7214168ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:31.528521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c9a7bd04-bcc8-4941-81c2-36c7214168ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:31.604071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:31.617821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7bd04-bcc8-4941-81c2-36c7214168ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:31.619407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7bd04-bcc8-4941-81c2-36c7214168ca service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:31.637337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 701/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:32.511008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a219008-a6ec-4de5-9237-146709df9231 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:32.511547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a219008-a6ec-4de5-9237-146709df9231 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:32.511547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a219008-a6ec-4de5-9237-146709df9231 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:32.512806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:32.517943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9d383-66e1-4495-acb8-c5bfd5727517 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:32.518373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9d383-66e1-4495-acb8-c5bfd5727517 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:32.518543 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9d383-66e1-4495-acb8-c5bfd5727517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:32.596742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a2b9d383-66e1-4495-acb8-c5bfd5727517 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:32.597990 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a2b9d383-66e1-4495-acb8-c5bfd5727517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:32.695543 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:32.850262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a219008-a6ec-4de5-9237-146709df9231 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:32.851879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3a219008-a6ec-4de5-9237-146709df9231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:32.936904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:32.999092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a219008-a6ec-4de5-9237-146709df9231 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:33.118915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b15bb-8ba3-4e44-be56-74843511495b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:33.119806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b15bb-8ba3-4e44-be56-74843511495b service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:33.120226 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b15bb-8ba3-4e44-be56-74843511495b service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:33.120634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b15bb-8ba3-4e44-be56-74843511495b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:33.123199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f69b15bb-8ba3-4e44-be56-74843511495b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:33.184281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:33.225121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b15bb-8ba3-4e44-be56-74843511495b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:33.226746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b15bb-8ba3-4e44-be56-74843511495b service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:33.245344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 726/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:34.145721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e0415-ce29-4ab7-96be-ae61f82bb1fe None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:34.146275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e0415-ce29-4ab7-96be-ae61f82bb1fe None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:34.146275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e0415-ce29-4ab7-96be-ae61f82bb1fe None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:34.147602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:34.152538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062308a-c18b-43bd-9780-34de3337c969 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:34.152963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062308a-c18b-43bd-9780-34de3337c969 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:34.153155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062308a-c18b-43bd-9780-34de3337c969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:34.228975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9062308a-c18b-43bd-9780-34de3337c969 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:34.230211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9062308a-c18b-43bd-9780-34de3337c969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:34.326609 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:34.479253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e0415-ce29-4ab7-96be-ae61f82bb1fe None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:34.480811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e08e0415-ce29-4ab7-96be-ae61f82bb1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:34.530117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:34.626696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e0415-ce29-4ab7-96be-ae61f82bb1fe None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:34.739525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca95ec-7763-4ecd-86b3-79d26ec7d22a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:34.749499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca95ec-7763-4ecd-86b3-79d26ec7d22a service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:34.749499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca95ec-7763-4ecd-86b3-79d26ec7d22a service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:34.749499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca95ec-7763-4ecd-86b3-79d26ec7d22a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:34.749499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-00ca95ec-7763-4ecd-86b3-79d26ec7d22a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:34.749499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:34.835992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca95ec-7763-4ecd-86b3-79d26ec7d22a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:34.837620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca95ec-7763-4ecd-86b3-79d26ec7d22a service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:34.855710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 704/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:35.711521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbe6b5-4f32-4ef2-9206-52ee4b5a3d40 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:35.712054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbe6b5-4f32-4ef2-9206-52ee4b5a3d40 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:35.712054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbe6b5-4f32-4ef2-9206-52ee4b5a3d40 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:35.713619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:35.718651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376a077-bc44-455d-866b-cfbf6e006976 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:35.719834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376a077-bc44-455d-866b-cfbf6e006976 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:35.719834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376a077-bc44-455d-866b-cfbf6e006976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:35.796209 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8376a077-bc44-455d-866b-cfbf6e006976 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:35.797426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8376a077-bc44-455d-866b-cfbf6e006976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:35.865873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:35.895017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:36.046633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbe6b5-4f32-4ef2-9206-52ee4b5a3d40 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:36.048446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-97fbe6b5-4f32-4ef2-9206-52ee4b5a3d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:36.066752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:36.195177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbe6b5-4f32-4ef2-9206-52ee4b5a3d40 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:36.280613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:36.305823 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ae2fe-9d05-47a6-a8f3-9a33f1e67036 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:36.306425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ae2fe-9d05-47a6-a8f3-9a33f1e67036 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:36.306704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ae2fe-9d05-47a6-a8f3-9a33f1e67036 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:36.306977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ae2fe-9d05-47a6-a8f3-9a33f1e67036 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:36.308579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-109ae2fe-9d05-47a6-a8f3-9a33f1e67036 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:36.399070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ae2fe-9d05-47a6-a8f3-9a33f1e67036 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:36.400672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ae2fe-9d05-47a6-a8f3-9a33f1e67036 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:36.418591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 729/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:37.280837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fbbc7-8be3-4ae4-9226-eff62e5e2f56 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:37.282058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fbbc7-8be3-4ae4-9226-eff62e5e2f56 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:37.282058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fbbc7-8be3-4ae4-9226-eff62e5e2f56 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:37.283382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:37.288312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaacde-926b-4785-b919-2e465f90af08 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:37.288774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaacde-926b-4785-b919-2e465f90af08 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:37.288885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaacde-926b-4785-b919-2e465f90af08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:37.366105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dbeaacde-926b-4785-b919-2e465f90af08 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:37.367323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dbeaacde-926b-4785-b919-2e465f90af08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:37.409226 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:37.463961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:37.613940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fbbc7-8be3-4ae4-9226-eff62e5e2f56 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:37.615469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f42fbbc7-8be3-4ae4-9226-eff62e5e2f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:37.762132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fbbc7-8be3-4ae4-9226-eff62e5e2f56 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:37.816686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:37.877427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7654f-0dcf-47c4-86c8-13c7dbb6dfde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:37.877983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7654f-0dcf-47c4-86c8-13c7dbb6dfde service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:37.878257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7654f-0dcf-47c4-86c8-13c7dbb6dfde service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:37.878524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7654f-0dcf-47c4-86c8-13c7dbb6dfde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:37.880108 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-eac7654f-0dcf-47c4-86c8-13c7dbb6dfde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:37.968664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7654f-0dcf-47c4-86c8-13c7dbb6dfde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:37.970334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7654f-0dcf-47c4-86c8-13c7dbb6dfde service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:37.988607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 707/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:38.864013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969370f-5d71-433b-898a-ba8aa37621f5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:38.864593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969370f-5d71-433b-898a-ba8aa37621f5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:38.864593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969370f-5d71-433b-898a-ba8aa37621f5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:38.865917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:38.871064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d95b-8497-4cbe-8848-662e28f88b25 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:38.871533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d95b-8497-4cbe-8848-662e28f88b25 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:38.871642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d95b-8497-4cbe-8848-662e28f88b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:38.948392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cd55d95b-8497-4cbe-8848-662e28f88b25 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:38.949620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cd55d95b-8497-4cbe-8848-662e28f88b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:38.951614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:39.046305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:39.201144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969370f-5d71-433b-898a-ba8aa37621f5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:39.202718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c969370f-5d71-433b-898a-ba8aa37621f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:39.350432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969370f-5d71-433b-898a-ba8aa37621f5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:39.378299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:39.464942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099fdc7-4d32-4451-98c3-3787f8ec110a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:39.465582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099fdc7-4d32-4451-98c3-3787f8ec110a service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:39.465862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099fdc7-4d32-4451-98c3-3787f8ec110a service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:39.466133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099fdc7-4d32-4451-98c3-3787f8ec110a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:39.467733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b099fdc7-4d32-4451-98c3-3787f8ec110a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:39.558115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099fdc7-4d32-4451-98c3-3787f8ec110a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:39.559710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099fdc7-4d32-4451-98c3-3787f8ec110a service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:39.578597 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 732/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:41.424610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715336de-3062-488b-a25a-9496abdd71e4 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:41.425176 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715336de-3062-488b-a25a-9496abdd71e4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:41.425176 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715336de-3062-488b-a25a-9496abdd71e4 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:41.426540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:41.431901 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da063c2a-e319-44c4-af22-026d892858a9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:41.432310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da063c2a-e319-44c4-af22-026d892858a9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:41.432433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da063c2a-e319-44c4-af22-026d892858a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:41.474312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:41.498851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-da063c2a-e319-44c4-af22-026d892858a9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:41.500581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-da063c2a-e319-44c4-af22-026d892858a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:41.598634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:41.900425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715336de-3062-488b-a25a-9496abdd71e4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:41.902006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-715336de-3062-488b-a25a-9496abdd71e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:42.005863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715336de-3062-488b-a25a-9496abdd71e4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:42.037427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:42.079164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b3553-f765-41f1-b823-106479a81ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:42.079765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b3553-f765-41f1-b823-106479a81ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:42.080063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b3553-f765-41f1-b823-106479a81ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:42.080334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b3553-f765-41f1-b823-106479a81ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:42.081955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-155b3553-f765-41f1-b823-106479a81ab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:42.195803 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b3553-f765-41f1-b823-106479a81ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:42.197992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b3553-f765-41f1-b823-106479a81ab0 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:42.224926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 710/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:44.408611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dee6b-85a4-4d77-b0cc-9cfe7c9d71b7 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:44.409204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dee6b-85a4-4d77-b0cc-9cfe7c9d71b7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:44.409204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dee6b-85a4-4d77-b0cc-9cfe7c9d71b7 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:44.410538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:24:44.415745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c934fcb-0c84-4b11-a797-579570fb3683 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:44.416130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c934fcb-0c84-4b11-a797-579570fb3683 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:44.416287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c934fcb-0c84-4b11-a797-579570fb3683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:44.491774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4c934fcb-0c84-4b11-a797-579570fb3683 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:44.492971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4c934fcb-0c84-4b11-a797-579570fb3683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:44.505586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:44.593406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:44.907469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7dee6b-85a4-4d77-b0cc-9cfe7c9d71b7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:44.909125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8b7dee6b-85a4-4d77-b0cc-9cfe7c9d71b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:45.014032 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7dee6b-85a4-4d77-b0cc-9cfe7c9d71b7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:45.026353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:45.088027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce6270-56b5-4cb7-b359-5d84cfb93974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:45.088631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce6270-56b5-4cb7-b359-5d84cfb93974 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:45.088912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce6270-56b5-4cb7-b359-5d84cfb93974 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:45.089184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce6270-56b5-4cb7-b359-5d84cfb93974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:45.090825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fbce6270-56b5-4cb7-b359-5d84cfb93974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:45.201050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce6270-56b5-4cb7-b359-5d84cfb93974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:45.202507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce6270-56b5-4cb7-b359-5d84cfb93974 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:45.220889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 735/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:47.498837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d68eee-7a57-415f-87d2-7c4ce882dcf3 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:47.499403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d68eee-7a57-415f-87d2-7c4ce882dcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:47.499403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d68eee-7a57-415f-87d2-7c4ce882dcf3 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:47.501355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:24:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:47.506468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae801cac-4994-49f5-8b3c-be8519a43af9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:47.506895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae801cac-4994-49f5-8b3c-be8519a43af9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:47.507009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae801cac-4994-49f5-8b3c-be8519a43af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:47.555297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:47.573747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ae801cac-4994-49f5-8b3c-be8519a43af9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:47.574965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ae801cac-4994-49f5-8b3c-be8519a43af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:47.670349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:47.962576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d68eee-7a57-415f-87d2-7c4ce882dcf3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:47.964104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d4d68eee-7a57-415f-87d2-7c4ce882dcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:48.067555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d68eee-7a57-415f-87d2-7c4ce882dcf3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:48.107997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:48.143590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec329606-ceba-45d7-b1ab-8626c783e19c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:48.144234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec329606-ceba-45d7-b1ab-8626c783e19c service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:48.144519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec329606-ceba-45d7-b1ab-8626c783e19c service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:48.144783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec329606-ceba-45d7-b1ab-8626c783e19c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:48.146413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ec329606-ceba-45d7-b1ab-8626c783e19c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:48.256131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec329606-ceba-45d7-b1ab-8626c783e19c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:48.257595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec329606-ceba-45d7-b1ab-8626c783e19c service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:48.275694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 713/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:50.311463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bc7ff-fb23-4e0e-a4d9-d5e1c0f81dc7 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:50.312534 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bc7ff-fb23-4e0e-a4d9-d5e1c0f81dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:50.312534 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bc7ff-fb23-4e0e-a4d9-d5e1c0f81dc7 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:50.313949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:50.319331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6724c-5a0e-4914-818f-49d8d6fb01fd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:50.319740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6724c-5a0e-4914-818f-49d8d6fb01fd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:50.319870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6724c-5a0e-4914-818f-49d8d6fb01fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:50.388850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5cb6724c-5a0e-4914-818f-49d8d6fb01fd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:50.390121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5cb6724c-5a0e-4914-818f-49d8d6fb01fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:50.410854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:50.488962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:50.776861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bc7ff-fb23-4e0e-a4d9-d5e1c0f81dc7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:50.778448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d16bc7ff-fb23-4e0e-a4d9-d5e1c0f81dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:50.881756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bc7ff-fb23-4e0e-a4d9-d5e1c0f81dc7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:50.901815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:50.956113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b7188-b6e9-4d66-82f2-33819c693660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:50.956718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b7188-b6e9-4d66-82f2-33819c693660 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:50.956993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b7188-b6e9-4d66-82f2-33819c693660 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:50.957260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b7188-b6e9-4d66-82f2-33819c693660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:50.958877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fa1b7188-b6e9-4d66-82f2-33819c693660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:51.066712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b7188-b6e9-4d66-82f2-33819c693660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:51.068056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b7188-b6e9-4d66-82f2-33819c693660 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:51.086273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 738/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:53.073147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bbcba-edc3-4395-9e32-6e237e7ec4dd None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:53.073699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bbcba-edc3-4395-9e32-6e237e7ec4dd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:53.073699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bbcba-edc3-4395-9e32-6e237e7ec4dd None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:53.074970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:53.081022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697de75e-f5a9-4e87-abfb-95b243ec7e2e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:53.081490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697de75e-f5a9-4e87-abfb-95b243ec7e2e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:53.081603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697de75e-f5a9-4e87-abfb-95b243ec7e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:53.146040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:53.148636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-697de75e-f5a9-4e87-abfb-95b243ec7e2e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:53.149903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-697de75e-f5a9-4e87-abfb-95b243ec7e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:53.246028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:53.533592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1bbcba-edc3-4395-9e32-6e237e7ec4dd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:53.535120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2e1bbcba-edc3-4395-9e32-6e237e7ec4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:53.638523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1bbcba-edc3-4395-9e32-6e237e7ec4dd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:53.687784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:53.714321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c1a8b-d360-4b75-a251-a810c47b46b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:53.714321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c1a8b-d360-4b75-a251-a810c47b46b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:53.714641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c1a8b-d360-4b75-a251-a810c47b46b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:53.714641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c1a8b-d360-4b75-a251-a810c47b46b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:53.716222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1e1c1a8b-d360-4b75-a251-a810c47b46b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:53.824350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c1a8b-d360-4b75-a251-a810c47b46b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:53.825810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c1a8b-d360-4b75-a251-a810c47b46b9 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:53.844911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 716/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:55.852711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425908f7-7b19-447a-b38e-665fe0ea0e14 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:55.853278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425908f7-7b19-447a-b38e-665fe0ea0e14 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:55.853278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425908f7-7b19-447a-b38e-665fe0ea0e14 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:55.854715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:55.860536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3f6fc-5c5a-4cc1-ba66-a74b96301306 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:55.860979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3f6fc-5c5a-4cc1-ba66-a74b96301306 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:55.861089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3f6fc-5c5a-4cc1-ba66-a74b96301306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:55.928497 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d7d3f6fc-5c5a-4cc1-ba66-a74b96301306 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:55.929747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d7d3f6fc-5c5a-4cc1-ba66-a74b96301306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:55.958900 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:56.026778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:56.316398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425908f7-7b19-447a-b38e-665fe0ea0e14 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:56.318010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-425908f7-7b19-447a-b38e-665fe0ea0e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:56.421685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425908f7-7b19-447a-b38e-665fe0ea0e14 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:56.450607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:56.495472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c73479-c445-4f38-81ee-b4356fde2cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:56.496082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c73479-c445-4f38-81ee-b4356fde2cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:56.496361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c73479-c445-4f38-81ee-b4356fde2cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:56.496630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c73479-c445-4f38-81ee-b4356fde2cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:56.498262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-57c73479-c445-4f38-81ee-b4356fde2cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:56.617905 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c73479-c445-4f38-81ee-b4356fde2cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:56.619331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c73479-c445-4f38-81ee-b4356fde2cb3 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:56.637666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 741/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:58.620890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed8122-d174-45d5-81e5-deabfe1c3dd2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:58.621481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed8122-d174-45d5-81e5-deabfe1c3dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:58.621481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed8122-d174-45d5-81e5-deabfe1c3dd2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:58.622810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:58.628003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a8b57-b606-4fb9-8ffe-887f81a612ce None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:58.628396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a8b57-b606-4fb9-8ffe-887f81a612ce None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:58.628568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a8b57-b606-4fb9-8ffe-887f81a612ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:58.697483 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ca1a8b57-b606-4fb9-8ffe-887f81a612ce None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:58.698722 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ca1a8b57-b606-4fb9-8ffe-887f81a612ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:58.707064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:58.799421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:59.091936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed8122-d174-45d5-81e5-deabfe1c3dd2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:59.094106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-76ed8122-d174-45d5-81e5-deabfe1c3dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:59.198417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed8122-d174-45d5-81e5-deabfe1c3dd2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:59.254121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:59.272206 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff796b3-f3dd-4710-9223-ede20136377f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:59.277315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff796b3-f3dd-4710-9223-ede20136377f service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:59.277315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff796b3-f3dd-4710-9223-ede20136377f service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:59.277315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff796b3-f3dd-4710-9223-ede20136377f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:59.277315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2ff796b3-f3dd-4710-9223-ede20136377f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:59.382472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff796b3-f3dd-4710-9223-ede20136377f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:59.383907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff796b3-f3dd-4710-9223-ede20136377f service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:59.402020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 719/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:01.373310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d1a03-a3b4-42f8-b2ed-ca04445721de None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:01.373976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d1a03-a3b4-42f8-b2ed-ca04445721de None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:01.373976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d1a03-a3b4-42f8-b2ed-ca04445721de None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:01.375439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:01.380911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf01854-63b3-4a1b-9b92-0952cde8a7bc None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:01.381375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf01854-63b3-4a1b-9b92-0952cde8a7bc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:01.381619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf01854-63b3-4a1b-9b92-0952cde8a7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:01.449158 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2cf01854-63b3-4a1b-9b92-0952cde8a7bc None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:01.450342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2cf01854-63b3-4a1b-9b92-0952cde8a7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:01.490898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:01.549979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:01.833910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d1a03-a3b4-42f8-b2ed-ca04445721de None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:01.835474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c05d1a03-a3b4-42f8-b2ed-ca04445721de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:01.940368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d1a03-a3b4-42f8-b2ed-ca04445721de None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:01.978546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:02.015123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365621bb-8d5e-4dd4-a365-da763c22fe71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:02.015720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365621bb-8d5e-4dd4-a365-da763c22fe71 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:02.015997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365621bb-8d5e-4dd4-a365-da763c22fe71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:02.016272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365621bb-8d5e-4dd4-a365-da763c22fe71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:02.017894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-365621bb-8d5e-4dd4-a365-da763c22fe71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:02.127290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365621bb-8d5e-4dd4-a365-da763c22fe71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:02.128713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365621bb-8d5e-4dd4-a365-da763c22fe71 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:02.146639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 744/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:04.146586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64157f-13ab-4f02-b0e4-7166113ce3e6 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:04.147171 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64157f-13ab-4f02-b0e4-7166113ce3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:04.147171 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64157f-13ab-4f02-b0e4-7166113ce3e6 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:04.148540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:04.153859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaf10c-ceb5-45d3-a7b5-c06e3de904fa None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:04.154283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaf10c-ceb5-45d3-a7b5-c06e3de904fa None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:04.154401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaf10c-ceb5-45d3-a7b5-c06e3de904fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:04.222327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-31eaf10c-ceb5-45d3-a7b5-c06e3de904fa None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:04.223539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-31eaf10c-ceb5-45d3-a7b5-c06e3de904fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:04.241583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:04.333013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:04.620160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64157f-13ab-4f02-b0e4-7166113ce3e6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:04.621736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8a64157f-13ab-4f02-b0e4-7166113ce3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:04.725792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64157f-13ab-4f02-b0e4-7166113ce3e6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:04.793346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:04.800763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cd0f8-3b89-4c5d-b0ec-7f1f31312226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:04.801403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cd0f8-3b89-4c5d-b0ec-7f1f31312226 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:04.801710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cd0f8-3b89-4c5d-b0ec-7f1f31312226 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:04.801982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cd0f8-3b89-4c5d-b0ec-7f1f31312226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:04.803608 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b63cd0f8-3b89-4c5d-b0ec-7f1f31312226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:04.917144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cd0f8-3b89-4c5d-b0ec-7f1f31312226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:04.918512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cd0f8-3b89-4c5d-b0ec-7f1f31312226 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:04.936690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 722/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:06.906513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659cbe-9a66-40da-af99-44034fadd400 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:06.907082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659cbe-9a66-40da-af99-44034fadd400 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:06.907082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e659cbe-9a66-40da-af99-44034fadd400 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:06.908387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:25:06.913710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12949abf-988e-4b62-aac3-d41295919359 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:06.914126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12949abf-988e-4b62-aac3-d41295919359 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:06.914254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12949abf-988e-4b62-aac3-d41295919359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:06.980071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-12949abf-988e-4b62-aac3-d41295919359 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:06.981188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-12949abf-988e-4b62-aac3-d41295919359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:07.028628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:07.079771 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:07.366774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659cbe-9a66-40da-af99-44034fadd400 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:07.368311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2e659cbe-9a66-40da-af99-44034fadd400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:07.471739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e659cbe-9a66-40da-af99-44034fadd400 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:07.518680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:07.544867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63da692-8483-4928-ab9b-b29259bf5f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:07.546094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63da692-8483-4928-ab9b-b29259bf5f69 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:07.546453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63da692-8483-4928-ab9b-b29259bf5f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:07.546725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63da692-8483-4928-ab9b-b29259bf5f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:07.548386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c63da692-8483-4928-ab9b-b29259bf5f69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:07.656547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63da692-8483-4928-ab9b-b29259bf5f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:07.657942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63da692-8483-4928-ab9b-b29259bf5f69 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:07.676706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 747/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:17.077957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=22 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:17.674484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=66 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:27.420037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb6fd4-6ee2-424c-b3ff-556e86b9bd83 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:27.420612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb6fd4-6ee2-424c-b3ff-556e86b9bd83 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:27.420612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb6fd4-6ee2-424c-b3ff-556e86b9bd83 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:27.421975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 724/1471] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:27.427219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9e710-f859-4824-a8da-1540bda679cf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:27.427637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9e710-f859-4824-a8da-1540bda679cf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:27.427804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9e710-f859-4824-a8da-1540bda679cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:27.493972 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e1b9e710-f859-4824-a8da-1540bda679cf None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:27.495203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e1b9e710-f859-4824-a8da-1540bda679cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:27.709731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 748/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:28.008178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f106c6b-d0fa-4515-a388-6aa3f5db1694 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:28.008463 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f106c6b-d0fa-4515-a388-6aa3f5db1694 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:28.008735 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f106c6b-d0fa-4515-a388-6aa3f5db1694 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:28.010107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 725/1473] 10.222.0.42 () {54 vars in 967 bytes} [Thu Sep 1 05:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:28.015138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d3ddc-e6d1-494e-8fbe-29b806869864 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:28.015427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d3ddc-e6d1-494e-8fbe-29b806869864 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:28.015704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d3ddc-e6d1-494e-8fbe-29b806869864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:28.093080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-009d3ddc-e6d1-494e-8fbe-29b806869864 None None] MFA Rules not processed for user `4c23597e0da2454abd2a5354d779b092`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:28.094326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-009d3ddc-e6d1-494e-8fbe-29b806869864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:28.115275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:28.172127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 749/1474] 10.222.0.42 () {58 vars in 1064 bytes} [Thu Sep 1 05:25:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:28.179561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f106c6b-d0fa-4515-a388-6aa3f5db1694 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:28.184662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2f106c6b-d0fa-4515-a388-6aa3f5db1694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:28.284062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f106c6b-d0fa-4515-a388-6aa3f5db1694 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:28.357356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32313575-00fe-4715-bef9-9890287bf7bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c23597e0da2454abd2a5354d779b092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:28.358269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32313575-00fe-4715-bef9-9890287bf7bb service glance] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:28.358543 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32313575-00fe-4715-bef9-9890287bf7bb service glance] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:28.358807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32313575-00fe-4715-bef9-9890287bf7bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:28.360385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-32313575-00fe-4715-bef9-9890287bf7bb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:28.413472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:28.469364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32313575-00fe-4715-bef9-9890287bf7bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:28.470837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32313575-00fe-4715-bef9-9890287bf7bb service glance] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:28.489812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 726/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:30.551838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd89de7-3b75-4ceb-8b43-0e3b0877361c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:30.552414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd89de7-3b75-4ceb-8b43-0e3b0877361c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:30.552414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd89de7-3b75-4ceb-8b43-0e3b0877361c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:30.553718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 750/1476] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:30.558810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fedb2-4ea2-4bee-a3a8-8cad6e228a66 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:30.559193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fedb2-4ea2-4bee-a3a8-8cad6e228a66 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:30.559356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fedb2-4ea2-4bee-a3a8-8cad6e228a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:30.625984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b78fedb2-4ea2-4bee-a3a8-8cad6e228a66 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:30.627124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b78fedb2-4ea2-4bee-a3a8-8cad6e228a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:30.726317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 727/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:31.003834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd89de7-3b75-4ceb-8b43-0e3b0877361c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:31.005315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5dd89de7-3b75-4ceb-8b43-0e3b0877361c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:31.130587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd89de7-3b75-4ceb-8b43-0e3b0877361c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:31.178928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:31.223751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2b3c1-5b83-4194-a67f-30ae0971cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:31.224359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b3c1-5b83-4194-a67f-30ae0971cfad admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:31.224635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b3c1-5b83-4194-a67f-30ae0971cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:31.224906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b3c1-5b83-4194-a67f-30ae0971cfad admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:31.225569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2b3c1-5b83-4194-a67f-30ae0971cfad admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:31.227291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-95e2b3c1-5b83-4194-a67f-30ae0971cfad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:25:31.227291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 01 05:25:31.227291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: Sep 01 05:25:31.228422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2b3c1-5b83-4194-a67f-30ae0971cfad admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:31.247949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 751/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 1 05:25:31 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:25:33.189614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37849cb8-9448-4b08-bf29-4ca8922305c7 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:33.190226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37849cb8-9448-4b08-bf29-4ca8922305c7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:33.190226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37849cb8-9448-4b08-bf29-4ca8922305c7 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:33.191522 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 728/1479] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:33.196939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e218-fb60-496a-92be-4a7d0069ef06 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:33.197370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e218-fb60-496a-92be-4a7d0069ef06 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:33.197523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e218-fb60-496a-92be-4a7d0069ef06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:33.263542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e852e218-fb60-496a-92be-4a7d0069ef06 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:33.264737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e852e218-fb60-496a-92be-4a7d0069ef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:33.361918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 752/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:25:33.654660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37849cb8-9448-4b08-bf29-4ca8922305c7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:33.656215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-37849cb8-9448-4b08-bf29-4ca8922305c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:33.666304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:33.805635 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37849cb8-9448-4b08-bf29-4ca8922305c7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:33.888537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:33.920120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d64b1-c363-43cc-b79a-dedd7f297212 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:33.920722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d64b1-c363-43cc-b79a-dedd7f297212 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:33.920998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d64b1-c363-43cc-b79a-dedd7f297212 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:33.921264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d64b1-c363-43cc-b79a-dedd7f297212 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:33.922887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-097d64b1-c363-43cc-b79a-dedd7f297212 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:34.014197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d64b1-c363-43cc-b79a-dedd7f297212 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:34.015790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d64b1-c363-43cc-b79a-dedd7f297212 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:34.034642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 729/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:36.131344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cf87f-2242-47c9-9120-d75eb0c338b1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:36.131903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cf87f-2242-47c9-9120-d75eb0c338b1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:36.131903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cf87f-2242-47c9-9120-d75eb0c338b1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:36.133233 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 753/1482] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:36.139067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f78d7f-ca6a-41b4-818e-305cbb5e52ea None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:36.139482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f78d7f-ca6a-41b4-818e-305cbb5e52ea None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:36.139592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f78d7f-ca6a-41b4-818e-305cbb5e52ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:36.206409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-44f78d7f-ca6a-41b4-818e-305cbb5e52ea None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:36.207621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-44f78d7f-ca6a-41b4-818e-305cbb5e52ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:36.253341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:36.302967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 730/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:36.600465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cf87f-2242-47c9-9120-d75eb0c338b1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:36.602037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d00cf87f-2242-47c9-9120-d75eb0c338b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:36.625364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:36.713577 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cf87f-2242-47c9-9120-d75eb0c338b1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:36.788026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f89e3f-54a6-44d8-80c8-d550f7b299e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:36.788640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f89e3f-54a6-44d8-80c8-d550f7b299e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:36.789038 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f89e3f-54a6-44d8-80c8-d550f7b299e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:36.789331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f89e3f-54a6-44d8-80c8-d550f7b299e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:36.790981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-04f89e3f-54a6-44d8-80c8-d550f7b299e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:36.867029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:36.902439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f89e3f-54a6-44d8-80c8-d550f7b299e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:36.904414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f89e3f-54a6-44d8-80c8-d550f7b299e3 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:36.923191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 754/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:38.894457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e7d09-fe6b-4ba4-8704-f7f5bc24a3e8 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:38.895057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e7d09-fe6b-4ba4-8704-f7f5bc24a3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:38.895057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e7d09-fe6b-4ba4-8704-f7f5bc24a3e8 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:38.896411 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 731/1485] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:38.901765 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be648504-b314-448b-a356-f882e1b4a766 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:38.902211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be648504-b314-448b-a356-f882e1b4a766 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:38.902327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be648504-b314-448b-a356-f882e1b4a766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:38.969805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-be648504-b314-448b-a356-f882e1b4a766 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:38.971026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-be648504-b314-448b-a356-f882e1b4a766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:39.066375 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 755/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:39.358210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e7d09-fe6b-4ba4-8704-f7f5bc24a3e8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:39.359745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3b3e7d09-fe6b-4ba4-8704-f7f5bc24a3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:39.465073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e7d09-fe6b-4ba4-8704-f7f5bc24a3e8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:39.537264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:39.539141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21bcc6-d3bb-44f7-98ed-6a27f11e882f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:39.539748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21bcc6-d3bb-44f7-98ed-6a27f11e882f service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:39.540025 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21bcc6-d3bb-44f7-98ed-6a27f11e882f service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:39.540302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21bcc6-d3bb-44f7-98ed-6a27f11e882f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:39.541935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fb21bcc6-d3bb-44f7-98ed-6a27f11e882f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:39.651474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21bcc6-d3bb-44f7-98ed-6a27f11e882f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:39.652863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21bcc6-d3bb-44f7-98ed-6a27f11e882f service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:39.671663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 732/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:41.787822 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429e9b1-7c74-46f7-8754-2bcd42be0b1b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:41.788372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429e9b1-7c74-46f7-8754-2bcd42be0b1b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:41.788372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429e9b1-7c74-46f7-8754-2bcd42be0b1b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:41.789736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 756/1488] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:41.795054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ac3cf-2348-4927-96d9-53c3dae80674 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:41.795516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ac3cf-2348-4927-96d9-53c3dae80674 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:41.795627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ac3cf-2348-4927-96d9-53c3dae80674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:41.863924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3a3ac3cf-2348-4927-96d9-53c3dae80674 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:41.865141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3a3ac3cf-2348-4927-96d9-53c3dae80674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:41.918400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:41.963046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 733/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:42.254470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429e9b1-7c74-46f7-8754-2bcd42be0b1b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:42.255997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1429e9b1-7c74-46f7-8754-2bcd42be0b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:42.292706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:42.360060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429e9b1-7c74-46f7-8754-2bcd42be0b1b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:42.430843 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd921728-1809-4a02-b9ea-0a4f65c41467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:42.431438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd921728-1809-4a02-b9ea-0a4f65c41467 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:42.431711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd921728-1809-4a02-b9ea-0a4f65c41467 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:42.431976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd921728-1809-4a02-b9ea-0a4f65c41467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:42.433571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cd921728-1809-4a02-b9ea-0a4f65c41467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:42.512939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:42.541713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd921728-1809-4a02-b9ea-0a4f65c41467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:42.543334 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd921728-1809-4a02-b9ea-0a4f65c41467 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:42.563007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 757/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:44.982465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff2e28-a00d-4c6a-abe7-e248d5e14893 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:44.982975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff2e28-a00d-4c6a-abe7-e248d5e14893 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:44.982975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff2e28-a00d-4c6a-abe7-e248d5e14893 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:44.984288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 734/1491] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 1 05:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:44.989661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b622bf-9354-48ce-8792-eab6b6c16a0f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:44.990140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b622bf-9354-48ce-8792-eab6b6c16a0f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:44.990248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b622bf-9354-48ce-8792-eab6b6c16a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:45.059718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-02b622bf-9354-48ce-8792-eab6b6c16a0f None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:45.061263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-02b622bf-9354-48ce-8792-eab6b6c16a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:45.159414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 758/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 1 05:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:45.452540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff2e28-a00d-4c6a-abe7-e248d5e14893 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:45.454412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-08ff2e28-a00d-4c6a-abe7-e248d5e14893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:45.559593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff2e28-a00d-4c6a-abe7-e248d5e14893 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:45.641360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaac72e-a67e-42e7-9cf0-bde2aef6bbf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:45.642262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaac72e-a67e-42e7-9cf0-bde2aef6bbf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:45.642731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaac72e-a67e-42e7-9cf0-bde2aef6bbf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:45.643189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaac72e-a67e-42e7-9cf0-bde2aef6bbf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:45.645571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1aaac72e-a67e-42e7-9cf0-bde2aef6bbf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:45.657256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:45.784107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaac72e-a67e-42e7-9cf0-bde2aef6bbf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:45.785648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaac72e-a67e-42e7-9cf0-bde2aef6bbf3 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:45.807628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 735/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:55.157570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=82 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:55.804082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=50 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:07.253032 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9551cb-15d2-49e3-be6b-bedb960b2dbe None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:07.253679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9551cb-15d2-49e3-be6b-bedb960b2dbe None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:07.253679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9551cb-15d2-49e3-be6b-bedb960b2dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:07.335842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-da9551cb-15d2-49e3-be6b-bedb960b2dbe None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:07.337328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-da9551cb-15d2-49e3-be6b-bedb960b2dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:07.433787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 759/1494] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:07.442062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e794a5-581e-416a-aeb7-b453a8fb6de7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:07.442536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e794a5-581e-416a-aeb7-b453a8fb6de7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:07.442642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e794a5-581e-416a-aeb7-b453a8fb6de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:07.518474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-29e794a5-581e-416a-aeb7-b453a8fb6de7 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:07.519633 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-29e794a5-581e-416a-aeb7-b453a8fb6de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:07.614797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 736/1495] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:07.620062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9551cb-15d2-49e3-be6b-bedb960b2dbe None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:07.621280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-da9551cb-15d2-49e3-be6b-bedb960b2dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:07.632783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9551cb-15d2-49e3-be6b-bedb960b2dbe None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:07.694382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:07.727909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:07.729069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:07.729069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:07.729069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:07.729732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:27:07.730039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:07.731050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:27:07.731050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 01 05:27:07.731050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: Sep 01 05:27:07.731947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:07.744330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 760/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:27:07.752304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e794a5-581e-416a-aeb7-b453a8fb6de7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:07.762632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e794a5-581e-416a-aeb7-b453a8fb6de7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:07.826376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:07.859732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7d86e-a672-4dbd-b64d-d0baab34c62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:07.860343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7d86e-a672-4dbd-b64d-d0baab34c62c admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:07.860619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7d86e-a672-4dbd-b64d-d0baab34c62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:07.860892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7d86e-a672-4dbd-b64d-d0baab34c62c admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:07.861560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7d86e-a672-4dbd-b64d-d0baab34c62c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:07.863801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-71d7d86e-a672-4dbd-b64d-d0baab34c62c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:27:07.863801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 01 05:27:07.863801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: Sep 01 05:27:07.865292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7d86e-a672-4dbd-b64d-d0baab34c62c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:07.984717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 737/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:27:07 2022] POST /identity/v3/projects => generated 379 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:27:07.988899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:07.990452 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:08.024227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985c63-781b-4d78-8d6e-511a55527358 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:08.118426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824fdc7-605b-4cb4-afdc-da0e1f99a485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:08.119035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824fdc7-605b-4cb4-afdc-da0e1f99a485 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:08.119311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824fdc7-605b-4cb4-afdc-da0e1f99a485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:08.119578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824fdc7-605b-4cb4-afdc-da0e1f99a485 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:08.120203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824fdc7-605b-4cb4-afdc-da0e1f99a485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:08.121241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-3824fdc7-605b-4cb4-afdc-da0e1f99a485 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:27:08.121241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 01 05:27:08.121241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: Sep 01 05:27:08.125545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824fdc7-605b-4cb4-afdc-da0e1f99a485 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:08.235381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 761/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:27:07 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:27:08.239530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7d86e-a672-4dbd-b64d-d0baab34c62c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:08.241040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-71d7d86e-a672-4dbd-b64d-d0baab34c62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:08.304069 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7d86e-a672-4dbd-b64d-d0baab34c62c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:08.388278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:08.398616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af0d0e-9f1e-4a37-88d4-8913576477fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:08.402242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af0d0e-9f1e-4a37-88d4-8913576477fe admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:08.402242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af0d0e-9f1e-4a37-88d4-8913576477fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:08.402242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af0d0e-9f1e-4a37-88d4-8913576477fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:08.402242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af0d0e-9f1e-4a37-88d4-8913576477fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:08.402242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-07af0d0e-9f1e-4a37-88d4-8913576477fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:27:08.402242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 01 05:27:08.402242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: Sep 01 05:27:08.402831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af0d0e-9f1e-4a37-88d4-8913576477fe admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:08.415602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 738/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:27:08.419107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824fdc7-605b-4cb4-afdc-da0e1f99a485 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:08.420647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3824fdc7-605b-4cb4-afdc-da0e1f99a485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:08.432631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824fdc7-605b-4cb4-afdc-da0e1f99a485 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:08.457982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:08.528013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9fc2e-e946-4e43-a64d-190d92693f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:08.528639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9fc2e-e946-4e43-a64d-190d92693f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:08.528917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9fc2e-e946-4e43-a64d-190d92693f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:08.529186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9fc2e-e946-4e43-a64d-190d92693f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:08.529866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9fc2e-e946-4e43-a64d-190d92693f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:08.530861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-fea9fc2e-e946-4e43-a64d-190d92693f15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:27:08.530861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 01 05:27:08.530861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: Sep 01 05:27:08.531745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9fc2e-e946-4e43-a64d-190d92693f15 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:08.545340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 762/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:27:08.549578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af0d0e-9f1e-4a37-88d4-8913576477fe admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:08.551097 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-07af0d0e-9f1e-4a37-88d4-8913576477fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:08.585354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af0d0e-9f1e-4a37-88d4-8913576477fe admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:08.678467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55020a6-dffb-47b9-9e01-ea5612005d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:08.679052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55020a6-dffb-47b9-9e01-ea5612005d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:08.679330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55020a6-dffb-47b9-9e01-ea5612005d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:08.679602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55020a6-dffb-47b9-9e01-ea5612005d36 admin admin] PATH_INFO: `/v3/projects/5286c1806ac04818acc13dbcbf701b86/users/559fdc7d7c5d4cbe9905ced8049820e8/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:08.702439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55020a6-dffb-47b9-9e01-ea5612005d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5286c1806ac04818acc13dbcbf701b86, user_id=559fdc7d7c5d4cbe9905ced8049820e8, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:08.703469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-c55020a6-dffb-47b9-9e01-ea5612005d36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:27:08.703469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 01 05:27:08.703469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: Sep 01 05:27:08.705412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55020a6-dffb-47b9-9e01-ea5612005d36 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:08.781622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 739/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:27:08 2022] PUT /identity/v3/projects/5286c1806ac04818acc13dbcbf701b86/users/559fdc7d7c5d4cbe9905ced8049820e8/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:27:08.794775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f854f-9df0-4b1d-983a-a65aea9b5092 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:08.795062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f854f-9df0-4b1d-983a-a65aea9b5092 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:08.795336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f854f-9df0-4b1d-983a-a65aea9b5092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:08.851437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7f3f854f-9df0-4b1d-983a-a65aea9b5092 None None] MFA Rules not processed for user `559fdc7d7c5d4cbe9905ced8049820e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:08.853326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7f3f854f-9df0-4b1d-983a-a65aea9b5092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:08.931408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 763/1502] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:08.944329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8d9ac-7af4-4cfb-9c53-fdb663e04470 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:08.944614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8d9ac-7af4-4cfb-9c53-fdb663e04470 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:08.944884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8d9ac-7af4-4cfb-9c53-fdb663e04470 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:08.946853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 740/1503] 10.222.0.42 () {54 vars in 964 bytes} [Thu Sep 1 05:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:08.951504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d23a-fa3b-4a4b-9651-ad592368db9f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:08.951786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d23a-fa3b-4a4b-9651-ad592368db9f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:08.952054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d23a-fa3b-4a4b-9651-ad592368db9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:08.984366 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:09.028870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1da0d23a-fa3b-4a4b-9651-ad592368db9f None None] MFA Rules not processed for user `7e0e8589931b46ca84168a2ad273b63d`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:09.030092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1da0d23a-fa3b-4a4b-9651-ad592368db9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:09.108250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 764/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Sep 1 05:27:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:09.118399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8d9ac-7af4-4cfb-9c53-fdb663e04470 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:09.119968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-95c8d9ac-7af4-4cfb-9c53-fdb663e04470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:09.170172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:09.222979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8d9ac-7af4-4cfb-9c53-fdb663e04470 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:09.295598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00109d8-d16c-4fbd-a8d1-8a823f5c6f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:09.296199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00109d8-d16c-4fbd-a8d1-8a823f5c6f9d service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:09.296477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00109d8-d16c-4fbd-a8d1-8a823f5c6f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:09.296751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00109d8-d16c-4fbd-a8d1-8a823f5c6f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:09.298381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f00109d8-d16c-4fbd-a8d1-8a823f5c6f9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:09.396591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00109d8-d16c-4fbd-a8d1-8a823f5c6f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:09.398067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00109d8-d16c-4fbd-a8d1-8a823f5c6f9d service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:09.402381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:09.416701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 741/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:27:09.463253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6715414-437e-47a8-bf0e-5f2a66633fd3 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:09.463680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6715414-437e-47a8-bf0e-5f2a66633fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:09.463793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6715414-437e-47a8-bf0e-5f2a66633fd3 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:09.465041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 765/1506] 10.222.0.42 () {52 vars in 830 bytes} [Thu Sep 1 05:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:09.468706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00109d8-d16c-4fbd-a8d1-8a823f5c6f9d service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:09.469951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f00109d8-d16c-4fbd-a8d1-8a823f5c6f9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:09.567084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00109d8-d16c-4fbd-a8d1-8a823f5c6f9d service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:09.661341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:09.661996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:09.662273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:09.662542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] PATH_INFO: `/v3/users/559fdc7d7c5d4cbe9905ced8049820e8` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:09.674505 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=559fdc7d7c5d4cbe9905ced8049820e8)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:09.675527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:27:09.675527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 01 05:27:09.675527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: Sep 01 05:27:09.676739 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:09.686842 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:09.901691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '559fdc7d7c5d4cbe9905ced8049820e8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:27:09.902311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '559fdc7d7c5d4cbe9905ced8049820e8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:27:09.985248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '559fdc7d7c5d4cbe9905ced8049820e8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:27:10.008303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '559fdc7d7c5d4cbe9905ced8049820e8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:27:10.022076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '559fdc7d7c5d4cbe9905ced8049820e8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:27:10.028161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 742/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:27:09 2022] DELETE /identity/v3/users/559fdc7d7c5d4cbe9905ced8049820e8 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:27:10.032487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fedce8-991a-4952-a328-4108d11015dc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:10.032771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fedce8-991a-4952-a328-4108d11015dc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:10.033048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fedce8-991a-4952-a328-4108d11015dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:10.090606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-49fedce8-991a-4952-a328-4108d11015dc None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:10.091798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-49fedce8-991a-4952-a328-4108d11015dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:10.167648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:10.180093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 766/1508] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:10.190012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:10.191241 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:10.291257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5d208-59b2-4597-acdc-031316694699 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:10.347456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:10.366006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f287e5c-9eda-463c-b938-398a9669529e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:10.366614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f287e5c-9eda-463c-b938-398a9669529e service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:10.366891 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f287e5c-9eda-463c-b938-398a9669529e service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:10.367160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f287e5c-9eda-463c-b938-398a9669529e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:10.368990 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8f287e5c-9eda-463c-b938-398a9669529e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:10.478819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f287e5c-9eda-463c-b938-398a9669529e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:10.480202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f287e5c-9eda-463c-b938-398a9669529e service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:10.500148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 743/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:27:10.962736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fedce8-991a-4952-a328-4108d11015dc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:10.963973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-49fedce8-991a-4952-a328-4108d11015dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:10.980942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fedce8-991a-4952-a328-4108d11015dc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:11.071746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:11.072381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:11.072655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:11.072921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] PATH_INFO: `/v3/projects/5286c1806ac04818acc13dbcbf701b86` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:11.082649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5286c1806ac04818acc13dbcbf701b86)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:11.083667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:27:11.083667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 01 05:27:11.083667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: Sep 01 05:27:11.084861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:11.238863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5286c1806ac04818acc13dbcbf701b86', 'request_id': 'req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:27:11.339471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5286c1806ac04818acc13dbcbf701b86', 'request_id': 'req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:27:11.353406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5286c1806ac04818acc13dbcbf701b86', 'request_id': 'req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:27:11.375915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] The token cache is being invalidate because project 5286c1806ac04818acc13dbcbf701b86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:27:11.381071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:27:11.381071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e46e996c-c992-4df7-a20a-2ab1f4bb1bd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:27:11.381071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 767/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:27:10 2022] DELETE /identity/v3/projects/5286c1806ac04818acc13dbcbf701b86 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:27:20.497731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=62 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:21.373981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:28:40.890225 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea700-ba09-421f-ac5b-4691d18dc5ad None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:40.890824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea700-ba09-421f-ac5b-4691d18dc5ad None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:40.890824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea700-ba09-421f-ac5b-4691d18dc5ad None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:40.892109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:28:40.897003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ce0cf-50f1-4d7b-99d7-81f27dc132bf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:40.897439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ce0cf-50f1-4d7b-99d7-81f27dc132bf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:40.897658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ce0cf-50f1-4d7b-99d7-81f27dc132bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:40.978677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fa5ce0cf-50f1-4d7b-99d7-81f27dc132bf None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:28:40.979953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fa5ce0cf-50f1-4d7b-99d7-81f27dc132bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:28:41.084854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:28:41.237843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ea700-ba09-421f-ac5b-4691d18dc5ad None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:28:41.240653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6c8ea700-ba09-421f-ac5b-4691d18dc5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:28:41.363943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ea700-ba09-421f-ac5b-4691d18dc5ad None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:28:41.370096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93209e5-8e3d-424b-a1c9-22b8cbfe2aa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:28:41.370693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93209e5-8e3d-424b-a1c9-22b8cbfe2aa0 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:41.370968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93209e5-8e3d-424b-a1c9-22b8cbfe2aa0 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:41.371237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93209e5-8e3d-424b-a1c9-22b8cbfe2aa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:41.379111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93209e5-8e3d-424b-a1c9-22b8cbfe2aa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:28:41.381874 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93209e5-8e3d-424b-a1c9-22b8cbfe2aa0 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:28:41.391425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 745/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:28:42.232914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611790ce-c43e-456a-b121-e71f3d5eb504 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:42.233514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611790ce-c43e-456a-b121-e71f3d5eb504 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:42.233514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611790ce-c43e-456a-b121-e71f3d5eb504 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:42.234829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:28:42.240003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e9eba-32d3-47de-9c1f-82b9db0ade61 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:42.240428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e9eba-32d3-47de-9c1f-82b9db0ade61 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:42.240556 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e9eba-32d3-47de-9c1f-82b9db0ade61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:42.260471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-069e9eba-32d3-47de-9c1f-82b9db0ade61 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:28:42.261726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-069e9eba-32d3-47de-9c1f-82b9db0ade61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:28:42.269478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:28:42.422785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611790ce-c43e-456a-b121-e71f3d5eb504 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:28:42.437991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611790ce-c43e-456a-b121-e71f3d5eb504 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:28:42.443762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e07723-fca5-4e18-81fa-b350f1ff6eb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:28:42.444359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e07723-fca5-4e18-81fa-b350f1ff6eb0 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:42.444639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e07723-fca5-4e18-81fa-b350f1ff6eb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:42.444912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e07723-fca5-4e18-81fa-b350f1ff6eb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:42.452856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e07723-fca5-4e18-81fa-b350f1ff6eb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:28:42.454531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e07723-fca5-4e18-81fa-b350f1ff6eb0 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:28:42.464016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 770/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:28:42.974485 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96984269-5a09-4542-aaed-8d2c20bdd1b3 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:42.974792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96984269-5a09-4542-aaed-8d2c20bdd1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:42.975029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96984269-5a09-4542-aaed-8d2c20bdd1b3 None None] PATH_INFO: `/v3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:42.976305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 747/1517] 10.222.0.46 () {54 vars in 925 bytes} [Thu Sep 1 05:28:42 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:28:42.982546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c9d1e-d0e0-4459-bd9d-1d4ccc21f85f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:42.982827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c9d1e-d0e0-4459-bd9d-1d4ccc21f85f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:42.983096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c9d1e-d0e0-4459-bd9d-1d4ccc21f85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:43.041316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-439c9d1e-d0e0-4459-bd9d-1d4ccc21f85f None None] MFA Rules not processed for user `c11c7fb612ab40d88f526b8b38dbfb23`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:28:43.042539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-439c9d1e-d0e0-4459-bd9d-1d4ccc21f85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:28:43.059795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:28:43.113475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 771/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Sep 1 05:28:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:28:52.255250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=56 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:28:52.266848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8f93d-b1ec-4516-a2ac-c63725d3e384 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:52.267128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8f93d-b1ec-4516-a2ac-c63725d3e384 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:52.267403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8f93d-b1ec-4516-a2ac-c63725d3e384 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:52.268752 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 748/1519] 10.222.0.42 () {54 vars in 961 bytes} [Thu Sep 1 05:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:28:52.274604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a841d3-2325-4fc2-bf78-32375f88c9b5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:52.275077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a841d3-2325-4fc2-bf78-32375f88c9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:52.275190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a841d3-2325-4fc2-bf78-32375f88c9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:52.294625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-76a841d3-2325-4fc2-bf78-32375f88c9b5 None None] MFA Rules not processed for user `c11c7fb612ab40d88f526b8b38dbfb23`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:28:52.295811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-76a841d3-2325-4fc2-bf78-32375f88c9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:28:52.304385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 772/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Sep 1 05:28:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:28:52.311822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8f93d-b1ec-4516-a2ac-c63725d3e384 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:28:52.325982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8f93d-b1ec-4516-a2ac-c63725d3e384 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:28:52.330880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f3b14-80bd-4196-9dfc-10d4c3b8586e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:28:52.331462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3b14-80bd-4196-9dfc-10d4c3b8586e service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:52.331745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3b14-80bd-4196-9dfc-10d4c3b8586e service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:52.332019 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3b14-80bd-4196-9dfc-10d4c3b8586e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:52.341012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f3b14-80bd-4196-9dfc-10d4c3b8586e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:28:52.342641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f3b14-80bd-4196-9dfc-10d4c3b8586e service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:28:52.352399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 749/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:28:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:28:53.724817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232b878-f51d-4181-a892-63c28a17d606 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:53.725423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232b878-f51d-4181-a892-63c28a17d606 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:53.725423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232b878-f51d-4181-a892-63c28a17d606 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:53.726729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:28:53.731671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b396-0300-49e8-8620-3686c983d63d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:53.732104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b396-0300-49e8-8620-3686c983d63d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:53.732202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b396-0300-49e8-8620-3686c983d63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:53.751375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3ba2b396-0300-49e8-8620-3686c983d63d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:28:53.752563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3ba2b396-0300-49e8-8620-3686c983d63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:28:53.761315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:28:53.914862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232b878-f51d-4181-a892-63c28a17d606 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:28:53.929742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232b878-f51d-4181-a892-63c28a17d606 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:28:53.935136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e458d8-66fc-46e7-bbb5-471663c2e0bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:28:53.935726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e458d8-66fc-46e7-bbb5-471663c2e0bc service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:53.936001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e458d8-66fc-46e7-bbb5-471663c2e0bc service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:53.936269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e458d8-66fc-46e7-bbb5-471663c2e0bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:53.945557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e458d8-66fc-46e7-bbb5-471663c2e0bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:28:53.947125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e458d8-66fc-46e7-bbb5-471663c2e0bc service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:28:53.956369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 774/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:28:54.792578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf46ac6-a104-4bf2-8a67-98faeecc89f2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:54.793116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf46ac6-a104-4bf2-8a67-98faeecc89f2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:54.793116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf46ac6-a104-4bf2-8a67-98faeecc89f2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:54.794532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:28:54.799186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea6a63-87f7-4f39-85b6-edddb6d8b7e0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:54.799616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea6a63-87f7-4f39-85b6-edddb6d8b7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:54.799767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea6a63-87f7-4f39-85b6-edddb6d8b7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:54.820278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-81ea6a63-87f7-4f39-85b6-edddb6d8b7e0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:28:54.821490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-81ea6a63-87f7-4f39-85b6-edddb6d8b7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:28:54.829250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:28:54.981799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf46ac6-a104-4bf2-8a67-98faeecc89f2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:28:54.996645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf46ac6-a104-4bf2-8a67-98faeecc89f2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:28:55.002054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a00538-ea56-4aff-80c4-dc86f3f66273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:28:55.002641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a00538-ea56-4aff-80c4-dc86f3f66273 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:55.002918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a00538-ea56-4aff-80c4-dc86f3f66273 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:55.003187 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a00538-ea56-4aff-80c4-dc86f3f66273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:55.011402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a00538-ea56-4aff-80c4-dc86f3f66273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:28:55.012987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a00538-ea56-4aff-80c4-dc86f3f66273 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:28:55.022577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 752/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:28:58.933382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390a0cd-59c3-4bcf-8ad6-de4e2898b7d4 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:58.933941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390a0cd-59c3-4bcf-8ad6-de4e2898b7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:58.933941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390a0cd-59c3-4bcf-8ad6-de4e2898b7d4 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:58.935288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:28:58.940590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284f846-0555-4873-87c9-33b226c15ae0 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:58.941026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284f846-0555-4873-87c9-33b226c15ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:58.941133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284f846-0555-4873-87c9-33b226c15ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:58.961712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3284f846-0555-4873-87c9-33b226c15ae0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:28:58.962940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3284f846-0555-4873-87c9-33b226c15ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:28:58.970812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:28:59.258847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390a0cd-59c3-4bcf-8ad6-de4e2898b7d4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:28:59.260551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d390a0cd-59c3-4bcf-8ad6-de4e2898b7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:28:59.338161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390a0cd-59c3-4bcf-8ad6-de4e2898b7d4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:28:59.341799 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1794fb-ca33-4a3c-aab1-fede91625f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:28:59.342408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1794fb-ca33-4a3c-aab1-fede91625f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:28:59.342687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1794fb-ca33-4a3c-aab1-fede91625f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:28:59.342962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1794fb-ca33-4a3c-aab1-fede91625f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:28:59.351906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1794fb-ca33-4a3c-aab1-fede91625f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:28:59.353330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1794fb-ca33-4a3c-aab1-fede91625f87 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:28:59.362696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 777/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:01.271811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec71d7c-0c9e-4e89-b028-b2bd51b74604 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:01.272376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec71d7c-0c9e-4e89-b028-b2bd51b74604 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:01.272376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec71d7c-0c9e-4e89-b028-b2bd51b74604 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:01.273768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:01.279345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a7f7b-308f-46b8-aa6f-7623338392b3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:01.279778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a7f7b-308f-46b8-aa6f-7623338392b3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:01.279875 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a7f7b-308f-46b8-aa6f-7623338392b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:01.299295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c14a7f7b-308f-46b8-aa6f-7623338392b3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:01.300484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c14a7f7b-308f-46b8-aa6f-7623338392b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:01.309068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:01.603589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec71d7c-0c9e-4e89-b028-b2bd51b74604 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:01.614995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec71d7c-0c9e-4e89-b028-b2bd51b74604 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:01.618608 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c219e-8e1d-4d1c-b5b8-6c884f2a9994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:01.619196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c219e-8e1d-4d1c-b5b8-6c884f2a9994 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:01.619475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c219e-8e1d-4d1c-b5b8-6c884f2a9994 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:01.619748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c219e-8e1d-4d1c-b5b8-6c884f2a9994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:01.629634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c219e-8e1d-4d1c-b5b8-6c884f2a9994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:01.640428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c219e-8e1d-4d1c-b5b8-6c884f2a9994 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:01.640707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 755/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:05.117913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68dc10-1127-467a-a016-e818fd79de19 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:05.118501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68dc10-1127-467a-a016-e818fd79de19 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:05.118501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68dc10-1127-467a-a016-e818fd79de19 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:05.119792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:05.124929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5733-0381-4898-b08d-665a90138833 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:05.125369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5733-0381-4898-b08d-665a90138833 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:05.125490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5733-0381-4898-b08d-665a90138833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:05.145535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7e5c5733-0381-4898-b08d-665a90138833 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:05.146726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7e5c5733-0381-4898-b08d-665a90138833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:05.154518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:05.711455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68dc10-1127-467a-a016-e818fd79de19 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:05.713158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cb68dc10-1127-467a-a016-e818fd79de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:05.779730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:05.792941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68dc10-1127-467a-a016-e818fd79de19 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:05.796592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f61ed-302e-41e2-b0ff-4e3170935237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:05.797180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f61ed-302e-41e2-b0ff-4e3170935237 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:05.797483 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f61ed-302e-41e2-b0ff-4e3170935237 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:05.797764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f61ed-302e-41e2-b0ff-4e3170935237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:05.806723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f61ed-302e-41e2-b0ff-4e3170935237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:05.808159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f61ed-302e-41e2-b0ff-4e3170935237 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:05.819283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 780/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:07.826445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad875e4-f91e-498f-98c4-8f15d4217b7f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:07.826971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad875e4-f91e-498f-98c4-8f15d4217b7f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:07.826971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad875e4-f91e-498f-98c4-8f15d4217b7f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:07.828259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:07.833470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c3cfa-1f9c-4047-9015-4d25a9a027d2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:07.833896 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c3cfa-1f9c-4047-9015-4d25a9a027d2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:07.834035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c3cfa-1f9c-4047-9015-4d25a9a027d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:07.853390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e32c3cfa-1f9c-4047-9015-4d25a9a027d2 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:07.854628 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e32c3cfa-1f9c-4047-9015-4d25a9a027d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:07.862453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:08.423551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad875e4-f91e-498f-98c4-8f15d4217b7f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:08.425264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fad875e4-f91e-498f-98c4-8f15d4217b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:08.439301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad875e4-f91e-498f-98c4-8f15d4217b7f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:08.442992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35489c-0c75-4fdd-8e86-0a11b0bd6fa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:08.443593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35489c-0c75-4fdd-8e86-0a11b0bd6fa5 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:08.443877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35489c-0c75-4fdd-8e86-0a11b0bd6fa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:08.444149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35489c-0c75-4fdd-8e86-0a11b0bd6fa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:08.453548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35489c-0c75-4fdd-8e86-0a11b0bd6fa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:08.454984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35489c-0c75-4fdd-8e86-0a11b0bd6fa5 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:08.465005 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 758/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:17.848048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=11 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:18.460884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=33 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:34.935746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90b5f6-cdda-43fc-a8de-52c74becb0d3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:34.936655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90b5f6-cdda-43fc-a8de-52c74becb0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:34.936655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90b5f6-cdda-43fc-a8de-52c74becb0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:34.950697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37544a-23d4-4c0e-ad55-97a1103aaf88 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:34.951272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37544a-23d4-4c0e-ad55-97a1103aaf88 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:34.951272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37544a-23d4-4c0e-ad55-97a1103aaf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:34.977704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9d90b5f6-cdda-43fc-a8de-52c74becb0d3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:34.978937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9d90b5f6-cdda-43fc-a8de-52c74becb0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:34.992603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 782/1540] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:34.994002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4b37544a-23d4-4c0e-ad55-97a1103aaf88 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:34.995464 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4b37544a-23d4-4c0e-ad55-97a1103aaf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.005387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc2497-858d-4428-9643-654b9846196e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.005752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc2497-858d-4428-9643-654b9846196e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.006034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc2497-858d-4428-9643-654b9846196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.022076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 759/1541] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:35.023782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453752f-24c9-4659-832c-df03dd11c5d1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.024300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453752f-24c9-4659-832c-df03dd11c5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.024799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453752f-24c9-4659-832c-df03dd11c5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.050063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-befc2497-858d-4428-9643-654b9846196e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.051411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-befc2497-858d-4428-9643-654b9846196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.059113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2453752f-24c9-4659-832c-df03dd11c5d1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.061054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2453752f-24c9-4659-832c-df03dd11c5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.070820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 760/1542] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.071347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 783/1543] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.072635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cdab1-e608-4e68-9503-3ce58fa3cb96 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.072920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cdab1-e608-4e68-9503-3ce58fa3cb96 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.073133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f37ff-f530-47f4-b9fc-9820ad11cf7b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.073263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cdab1-e608-4e68-9503-3ce58fa3cb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.077887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f37ff-f530-47f4-b9fc-9820ad11cf7b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.078428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f37ff-f530-47f4-b9fc-9820ad11cf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.109567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-147cdab1-e608-4e68-9503-3ce58fa3cb96 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.111331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ee5f37ff-f530-47f4-b9fc-9820ad11cf7b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.111548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-147cdab1-e608-4e68-9503-3ce58fa3cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.113161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ee5f37ff-f530-47f4-b9fc-9820ad11cf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.121069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 784/1544] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.121984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cdab1-e608-4e68-9503-3ce58fa3cb96 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.123205 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 761/1545] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.124751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba420711-1c5c-4125-9751-bfd7d0e75545 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.125289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba420711-1c5c-4125-9751-bfd7d0e75545 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.125832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba420711-1c5c-4125-9751-bfd7d0e75545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.139033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cdab1-e608-4e68-9503-3ce58fa3cb96 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.146281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71489a4-40a1-451f-89c4-6b678f534aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.146924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71489a4-40a1-451f-89c4-6b678f534aea admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.147205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71489a4-40a1-451f-89c4-6b678f534aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.147478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71489a4-40a1-451f-89c4-6b678f534aea admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.148146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71489a4-40a1-451f-89c4-6b678f534aea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:35.148446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71489a4-40a1-451f-89c4-6b678f534aea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.150337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71489a4-40a1-451f-89c4-6b678f534aea admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.156803 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ba420711-1c5c-4125-9751-bfd7d0e75545 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.159562 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ba420711-1c5c-4125-9751-bfd7d0e75545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.168644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 762/1546] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.175374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed582e2-cf3a-4f27-942b-f2c81fcc0fa7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.177116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed582e2-cf3a-4f27-942b-f2c81fcc0fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.177654 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed582e2-cf3a-4f27-942b-f2c81fcc0fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.182534 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 785/1547] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:35.184045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69b3c6-0ec0-49da-9d88-ad017c820f76 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.185089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69b3c6-0ec0-49da-9d88-ad017c820f76 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.186464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69b3c6-0ec0-49da-9d88-ad017c820f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.219055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3ed582e2-cf3a-4f27-942b-f2c81fcc0fa7 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.220585 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3ed582e2-cf3a-4f27-942b-f2c81fcc0fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.226443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5a69b3c6-0ec0-49da-9d88-ad017c820f76 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.228119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5a69b3c6-0ec0-49da-9d88-ad017c820f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.238099 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 763/1548] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.241622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed582e2-cf3a-4f27-942b-f2c81fcc0fa7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.245255 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 786/1549] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.246809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dce7a7-27d9-445c-8a9f-2b3ff33c2153 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.247326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dce7a7-27d9-445c-8a9f-2b3ff33c2153 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.247909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dce7a7-27d9-445c-8a9f-2b3ff33c2153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.264135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed582e2-cf3a-4f27-942b-f2c81fcc0fa7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.271347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaac3d-1109-4d4a-9674-d67fe2b8fef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.271967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaac3d-1109-4d4a-9674-d67fe2b8fef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.272247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaac3d-1109-4d4a-9674-d67fe2b8fef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.272523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaac3d-1109-4d4a-9674-d67fe2b8fef3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.273180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaac3d-1109-4d4a-9674-d67fe2b8fef3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:35.276192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c4dce7a7-27d9-445c-8a9f-2b3ff33c2153 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.277683 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c4dce7a7-27d9-445c-8a9f-2b3ff33c2153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.278159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaac3d-1109-4d4a-9674-d67fe2b8fef3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.279249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-fcaaac3d-1109-4d4a-9674-d67fe2b8fef3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:30:35.279249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 01 05:30:35.279249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: Sep 01 05:30:35.280248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaac3d-1109-4d4a-9674-d67fe2b8fef3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.289030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 787/1550] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.291197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4798a-3d85-4385-b028-54ebcabd04ab None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.291723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4798a-3d85-4385-b028-54ebcabd04ab None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.292218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4798a-3d85-4385-b028-54ebcabd04ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.296471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 764/1551] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:35.302234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaac3d-1109-4d4a-9674-d67fe2b8fef3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.314259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaac3d-1109-4d4a-9674-d67fe2b8fef3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.324425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028d464-cc58-4423-913a-6da9ecd6c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.324834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-91d4798a-3d85-4385-b028-54ebcabd04ab None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.325653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028d464-cc58-4423-913a-6da9ecd6c97c admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.326138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028d464-cc58-4423-913a-6da9ecd6c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.326269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-91d4798a-3d85-4385-b028-54ebcabd04ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.326431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028d464-cc58-4423-913a-6da9ecd6c97c admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.333056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028d464-cc58-4423-913a-6da9ecd6c97c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.333056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028d464-cc58-4423-913a-6da9ecd6c97c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.335275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 788/1552] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.339882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622ce02-a79a-4d8e-a4a1-45eea976960e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.340202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622ce02-a79a-4d8e-a4a1-45eea976960e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.340439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622ce02-a79a-4d8e-a4a1-45eea976960e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.366502 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1622ce02-a79a-4d8e-a4a1-45eea976960e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.376390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1622ce02-a79a-4d8e-a4a1-45eea976960e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.377056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 789/1553] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.377882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622ce02-a79a-4d8e-a4a1-45eea976960e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.388694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622ce02-a79a-4d8e-a4a1-45eea976960e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.394917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da01480-da16-48b0-a9e6-5b80a209696b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.395625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da01480-da16-48b0-a9e6-5b80a209696b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.395828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da01480-da16-48b0-a9e6-5b80a209696b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.397533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da01480-da16-48b0-a9e6-5b80a209696b admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.397533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da01480-da16-48b0-a9e6-5b80a209696b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:35.397533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da01480-da16-48b0-a9e6-5b80a209696b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.399069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da01480-da16-48b0-a9e6-5b80a209696b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.412859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 790/1554] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:35.414215 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb5cc2-8940-4477-9b24-5c7d48a511c0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.414508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb5cc2-8940-4477-9b24-5c7d48a511c0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.414781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb5cc2-8940-4477-9b24-5c7d48a511c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.435465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-53fb5cc2-8940-4477-9b24-5c7d48a511c0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.436924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-53fb5cc2-8940-4477-9b24-5c7d48a511c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.445807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 791/1555] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.446484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb5cc2-8940-4477-9b24-5c7d48a511c0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.458871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb5cc2-8940-4477-9b24-5c7d48a511c0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.466861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 765/1556] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:35.466861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028d464-cc58-4423-913a-6da9ecd6c97c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.526729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f06ff9-acea-4612-bc63-90ad0d964cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.527292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f06ff9-acea-4612-bc63-90ad0d964cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.531616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f06ff9-acea-4612-bc63-90ad0d964cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.531616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f06ff9-acea-4612-bc63-90ad0d964cbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.531616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f06ff9-acea-4612-bc63-90ad0d964cbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:35.531616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f06ff9-acea-4612-bc63-90ad0d964cbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.532727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f06ff9-acea-4612-bc63-90ad0d964cbc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.538797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028d464-cc58-4423-913a-6da9ecd6c97c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.545705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78c040-3c77-45c8-a400-8dcf613a7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.546330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78c040-3c77-45c8-a400-8dcf613a7836 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.546609 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78c040-3c77-45c8-a400-8dcf613a7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.547252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78c040-3c77-45c8-a400-8dcf613a7836 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.547812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78c040-3c77-45c8-a400-8dcf613a7836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.548056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 792/1557] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:35.549555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00a14-d6e0-4a58-950f-27104b799e3d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.549751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78c040-3c77-45c8-a400-8dcf613a7836 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.549854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00a14-d6e0-4a58-950f-27104b799e3d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.550106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00a14-d6e0-4a58-950f-27104b799e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.575298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2ba00a14-d6e0-4a58-950f-27104b799e3d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.576576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2ba00a14-d6e0-4a58-950f-27104b799e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.585347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 793/1558] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.586288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00a14-d6e0-4a58-950f-27104b799e3d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.596895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00a14-d6e0-4a58-950f-27104b799e3d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.601641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c97ae-7865-4d5f-b1c0-0bf24ab39076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.602242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c97ae-7865-4d5f-b1c0-0bf24ab39076 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.602521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c97ae-7865-4d5f-b1c0-0bf24ab39076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.602793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c97ae-7865-4d5f-b1c0-0bf24ab39076 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.604109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c97ae-7865-4d5f-b1c0-0bf24ab39076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:35.604109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c97ae-7865-4d5f-b1c0-0bf24ab39076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.605481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c97ae-7865-4d5f-b1c0-0bf24ab39076 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.617622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 794/1559] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:35.622470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c97ae-7865-4d5f-b1c0-0bf24ab39076 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.636272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c97ae-7865-4d5f-b1c0-0bf24ab39076 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.644467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd11f86-7d14-4ae0-85da-4fbc60ffb42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.645087 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd11f86-7d14-4ae0-85da-4fbc60ffb42e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.645365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd11f86-7d14-4ae0-85da-4fbc60ffb42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.645831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd11f86-7d14-4ae0-85da-4fbc60ffb42e admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.646482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd11f86-7d14-4ae0-85da-4fbc60ffb42e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:35.647087 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd11f86-7d14-4ae0-85da-4fbc60ffb42e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.648688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd11f86-7d14-4ae0-85da-4fbc60ffb42e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.662208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 795/1560] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:35.663333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd11f86-7d14-4ae0-85da-4fbc60ffb42e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.674821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd11f86-7d14-4ae0-85da-4fbc60ffb42e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.679667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171848b-85c9-4ebe-bdce-9d8a7d13678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.685616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171848b-85c9-4ebe-bdce-9d8a7d13678f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.685616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171848b-85c9-4ebe-bdce-9d8a7d13678f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.685616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171848b-85c9-4ebe-bdce-9d8a7d13678f admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.685616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171848b-85c9-4ebe-bdce-9d8a7d13678f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.685616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-4171848b-85c9-4ebe-bdce-9d8a7d13678f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:30:35.685616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 01 05:30:35.685616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: Sep 01 05:30:35.686469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 766/1561] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:35.686562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171848b-85c9-4ebe-bdce-9d8a7d13678f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.686639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6af674-2257-4a15-8a05-36db9bcf29b9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.686639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6af674-2257-4a15-8a05-36db9bcf29b9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.686639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6af674-2257-4a15-8a05-36db9bcf29b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.711592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5e6af674-2257-4a15-8a05-36db9bcf29b9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:35.712909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5e6af674-2257-4a15-8a05-36db9bcf29b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.779431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 767/1562] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:35.780535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6af674-2257-4a15-8a05-36db9bcf29b9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.790930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 796/1563] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:35.790930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171848b-85c9-4ebe-bdce-9d8a7d13678f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.799323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6af674-2257-4a15-8a05-36db9bcf29b9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.799323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:35.821450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:35.853970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e27bd-ccdd-46c9-a8b3-4314573e2835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.854869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e27bd-ccdd-46c9-a8b3-4314573e2835 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.855368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e27bd-ccdd-46c9-a8b3-4314573e2835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.855638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e27bd-ccdd-46c9-a8b3-4314573e2835 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.856532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e27bd-ccdd-46c9-a8b3-4314573e2835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.857321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-524e27bd-ccdd-46c9-a8b3-4314573e2835 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:30:35.857321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 01 05:30:35.857321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: Sep 01 05:30:35.858516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e27bd-ccdd-46c9-a8b3-4314573e2835 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.867744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171848b-85c9-4ebe-bdce-9d8a7d13678f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.873559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dba1af-edc6-40a4-b3dd-b1edd5b9147c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.880068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dba1af-edc6-40a4-b3dd-b1edd5b9147c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.880068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dba1af-edc6-40a4-b3dd-b1edd5b9147c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.880068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dba1af-edc6-40a4-b3dd-b1edd5b9147c admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.880068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dba1af-edc6-40a4-b3dd-b1edd5b9147c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.880068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dba1af-edc6-40a4-b3dd-b1edd5b9147c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.982824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 768/1564] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:35.984091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19f69a-19c0-411f-a316-0638ee56a32d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.984377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19f69a-19c0-411f-a316-0638ee56a32d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.984648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19f69a-19c0-411f-a316-0638ee56a32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.004425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 797/1565] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:36.004425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dba1af-edc6-40a4-b3dd-b1edd5b9147c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.008574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3d19f69a-19c0-411f-a316-0638ee56a32d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:36.009984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3d19f69a-19c0-411f-a316-0638ee56a32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:36.076220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dba1af-edc6-40a4-b3dd-b1edd5b9147c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.082067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556d666-159a-4bac-861e-038df3e237df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.082697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556d666-159a-4bac-861e-038df3e237df admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.082966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556d666-159a-4bac-861e-038df3e237df admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.083332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556d666-159a-4bac-861e-038df3e237df admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.083969 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556d666-159a-4bac-861e-038df3e237df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.085931 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556d666-159a-4bac-861e-038df3e237df admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.095673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 769/1566] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:36.096481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19f69a-19c0-411f-a316-0638ee56a32d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.106316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19f69a-19c0-411f-a316-0638ee56a32d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.111739 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b6bd6-d664-4ad3-9930-61c03a8463ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.112369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6bd6-d664-4ad3-9930-61c03a8463ad admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.112646 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6bd6-d664-4ad3-9930-61c03a8463ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.112919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6bd6-d664-4ad3-9930-61c03a8463ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.113696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b6bd6-d664-4ad3-9930-61c03a8463ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.115650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b6bd6-d664-4ad3-9930-61c03a8463ad admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.216933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 798/1567] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:36 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:36.216933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556d666-159a-4bac-861e-038df3e237df admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.270576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 770/1568] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:36 2022] POST /identity/v3/projects => generated 393 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:36.271952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b6bd6-d664-4ad3-9930-61c03a8463ad admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.282989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556d666-159a-4bac-861e-038df3e237df admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.321202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:36.344698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1e91f-c29a-486a-88af-de8a5385a529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.345416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e91f-c29a-486a-88af-de8a5385a529 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.345817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e91f-c29a-486a-88af-de8a5385a529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.346207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e91f-c29a-486a-88af-de8a5385a529 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.346977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1e91f-c29a-486a-88af-de8a5385a529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.349121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1e91f-c29a-486a-88af-de8a5385a529 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.353778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b6bd6-d664-4ad3-9930-61c03a8463ad admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.359106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060dfa6a-35d2-49fc-b68d-b3b85c63781d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.359591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dfa6a-35d2-49fc-b68d-b3b85c63781d admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.360105 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dfa6a-35d2-49fc-b68d-b3b85c63781d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.360676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dfa6a-35d2-49fc-b68d-b3b85c63781d admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.361404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060dfa6a-35d2-49fc-b68d-b3b85c63781d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:36.365178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060dfa6a-35d2-49fc-b68d-b3b85c63781d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.365178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060dfa6a-35d2-49fc-b68d-b3b85c63781d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.376748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 771/1569] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:36.377687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060dfa6a-35d2-49fc-b68d-b3b85c63781d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.388903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060dfa6a-35d2-49fc-b68d-b3b85c63781d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.394408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc632e7-ace5-497c-a69e-7b896d772e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.395030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc632e7-ace5-497c-a69e-7b896d772e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.395315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc632e7-ace5-497c-a69e-7b896d772e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.395592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc632e7-ace5-497c-a69e-7b896d772e4b admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.396227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc632e7-ace5-497c-a69e-7b896d772e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.398479 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc632e7-ace5-497c-a69e-7b896d772e4b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.410160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:36.486967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 799/1570] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:36 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:36.487842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1e91f-c29a-486a-88af-de8a5385a529 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.498002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1e91f-c29a-486a-88af-de8a5385a529 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.504162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c75df1-9bb4-40a7-8b08-5f7d846dd2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.504831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c75df1-9bb4-40a7-8b08-5f7d846dd2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.505162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c75df1-9bb4-40a7-8b08-5f7d846dd2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.505389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c75df1-9bb4-40a7-8b08-5f7d846dd2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.506142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c75df1-9bb4-40a7-8b08-5f7d846dd2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.507738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c75df1-9bb4-40a7-8b08-5f7d846dd2b0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.519541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 800/1571] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:36.520724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c75df1-9bb4-40a7-8b08-5f7d846dd2b0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.521244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 772/1572] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:36 2022] POST /identity/v3/users => generated 284 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:36.521973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc632e7-ace5-497c-a69e-7b896d772e4b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.536018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc632e7-ace5-497c-a69e-7b896d772e4b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.536919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c75df1-9bb4-40a7-8b08-5f7d846dd2b0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.540774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d63a3-a642-4d99-b19a-c7a5187b5e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.541378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d63a3-a642-4d99-b19a-c7a5187b5e84 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.549046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d63a3-a642-4d99-b19a-c7a5187b5e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.549046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d63a3-a642-4d99-b19a-c7a5187b5e84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.549294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c80a9-ee2b-44b3-8f8e-bdf69a342f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.549294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c80a9-ee2b-44b3-8f8e-bdf69a342f98 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.549528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d63a3-a642-4d99-b19a-c7a5187b5e84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:36.549640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c80a9-ee2b-44b3-8f8e-bdf69a342f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.549737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d63a3-a642-4d99-b19a-c7a5187b5e84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.549833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c80a9-ee2b-44b3-8f8e-bdf69a342f98 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.549833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c80a9-ee2b-44b3-8f8e-bdf69a342f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.550002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d63a3-a642-4d99-b19a-c7a5187b5e84 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.550097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c80a9-ee2b-44b3-8f8e-bdf69a342f98 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.564516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 773/1573] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:36.565524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d63a3-a642-4d99-b19a-c7a5187b5e84 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.576750 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d63a3-a642-4d99-b19a-c7a5187b5e84 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.582400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b5abd-abc9-4136-84b9-8f83b050b335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.582993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b5abd-abc9-4136-84b9-8f83b050b335 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.583274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b5abd-abc9-4136-84b9-8f83b050b335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.583554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b5abd-abc9-4136-84b9-8f83b050b335 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.584179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b5abd-abc9-4136-84b9-8f83b050b335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.586160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b5abd-abc9-4136-84b9-8f83b050b335 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.686435 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 801/1574] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:36 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:36.687193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c80a9-ee2b-44b3-8f8e-bdf69a342f98 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.700364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c80a9-ee2b-44b3-8f8e-bdf69a342f98 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.705257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e97b5-0c7f-4694-a8f1-5e0b6944fb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.706019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e97b5-0c7f-4694-a8f1-5e0b6944fb4d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.706310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e97b5-0c7f-4694-a8f1-5e0b6944fb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.706582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e97b5-0c7f-4694-a8f1-5e0b6944fb4d admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.707217 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e97b5-0c7f-4694-a8f1-5e0b6944fb4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.709293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e97b5-0c7f-4694-a8f1-5e0b6944fb4d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.716920 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 774/1575] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:36 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:36.717596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b5abd-abc9-4136-84b9-8f83b050b335 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.730025 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b5abd-abc9-4136-84b9-8f83b050b335 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.736055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbecef2d-ca79-4b95-a553-a7906ed7088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.742002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecef2d-ca79-4b95-a553-a7906ed7088e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.742002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecef2d-ca79-4b95-a553-a7906ed7088e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.742002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecef2d-ca79-4b95-a553-a7906ed7088e admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.742002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbecef2d-ca79-4b95-a553-a7906ed7088e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.742002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbecef2d-ca79-4b95-a553-a7906ed7088e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.822562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 802/1576] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:36 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:36.823698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e97b5-0c7f-4694-a8f1-5e0b6944fb4d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.835176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e97b5-0c7f-4694-a8f1-5e0b6944fb4d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.842864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 775/1577] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:36 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:36.843028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706f1ab-6a58-4339-8128-dfdeabe2aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.843163 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbecef2d-ca79-4b95-a553-a7906ed7088e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.843262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706f1ab-6a58-4339-8128-dfdeabe2aa24 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.843262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706f1ab-6a58-4339-8128-dfdeabe2aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.843262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706f1ab-6a58-4339-8128-dfdeabe2aa24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.843262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706f1ab-6a58-4339-8128-dfdeabe2aa24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.846243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706f1ab-6a58-4339-8128-dfdeabe2aa24 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.860006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 803/1578] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:36.860006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706f1ab-6a58-4339-8128-dfdeabe2aa24 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.868648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706f1ab-6a58-4339-8128-dfdeabe2aa24 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.914703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbecef2d-ca79-4b95-a553-a7906ed7088e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.921123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b279fc-e942-47a3-8ee7-63b19206f791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.921905 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b279fc-e942-47a3-8ee7-63b19206f791 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.922254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b279fc-e942-47a3-8ee7-63b19206f791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.922560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b279fc-e942-47a3-8ee7-63b19206f791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.923223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b279fc-e942-47a3-8ee7-63b19206f791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.925324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b279fc-e942-47a3-8ee7-63b19206f791 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.933000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123485eb-d63c-429e-b227-4b6a0d009da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.933665 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123485eb-d63c-429e-b227-4b6a0d009da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.934128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123485eb-d63c-429e-b227-4b6a0d009da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.934372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123485eb-d63c-429e-b227-4b6a0d009da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.935108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123485eb-d63c-429e-b227-4b6a0d009da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.940510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 776/1579] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:36.940670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123485eb-d63c-429e-b227-4b6a0d009da0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.940768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b279fc-e942-47a3-8ee7-63b19206f791 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.951864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 804/1580] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:36.951864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123485eb-d63c-429e-b227-4b6a0d009da0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.952162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b279fc-e942-47a3-8ee7-63b19206f791 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.958856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442377e-c67f-4ed0-b6a6-363a0ca9c47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.960113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442377e-c67f-4ed0-b6a6-363a0ca9c47c admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.960395 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442377e-c67f-4ed0-b6a6-363a0ca9c47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.960845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442377e-c67f-4ed0-b6a6-363a0ca9c47c admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.961586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442377e-c67f-4ed0-b6a6-363a0ca9c47c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.963147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123485eb-d63c-429e-b227-4b6a0d009da0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.963507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442377e-c67f-4ed0-b6a6-363a0ca9c47c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.968083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d0f60-a487-4fbd-b43a-033378e2c0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.968691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d0f60-a487-4fbd-b43a-033378e2c0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.969026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d0f60-a487-4fbd-b43a-033378e2c0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.969294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d0f60-a487-4fbd-b43a-033378e2c0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.970384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d0f60-a487-4fbd-b43a-033378e2c0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.971996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d0f60-a487-4fbd-b43a-033378e2c0b0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.982983 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 805/1581] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:36.982983 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d0f60-a487-4fbd-b43a-033378e2c0b0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.993729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d0f60-a487-4fbd-b43a-033378e2c0b0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.998602 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b7f12-bc89-475e-a0c0-fe7de39708db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.999229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b7f12-bc89-475e-a0c0-fe7de39708db admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.999482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b7f12-bc89-475e-a0c0-fe7de39708db admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.999756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b7f12-bc89-475e-a0c0-fe7de39708db admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.000388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b7f12-bc89-475e-a0c0-fe7de39708db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.002587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b7f12-bc89-475e-a0c0-fe7de39708db admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.012476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 806/1582] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.013418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b7f12-bc89-475e-a0c0-fe7de39708db admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.023967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b7f12-bc89-475e-a0c0-fe7de39708db admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.029522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92615452-946a-4a57-95ef-81c66410d40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.030123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615452-946a-4a57-95ef-81c66410d40b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.030400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615452-946a-4a57-95ef-81c66410d40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.030669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615452-946a-4a57-95ef-81c66410d40b admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.031304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92615452-946a-4a57-95ef-81c66410d40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.033271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92615452-946a-4a57-95ef-81c66410d40b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.044364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 807/1583] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.045268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92615452-946a-4a57-95ef-81c66410d40b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.056067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92615452-946a-4a57-95ef-81c66410d40b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.061691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067d4fd-913c-48da-bb3b-47e60f79ae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.062337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067d4fd-913c-48da-bb3b-47e60f79ae48 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.062620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067d4fd-913c-48da-bb3b-47e60f79ae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.063484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067d4fd-913c-48da-bb3b-47e60f79ae48 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.064157 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067d4fd-913c-48da-bb3b-47e60f79ae48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.066302 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067d4fd-913c-48da-bb3b-47e60f79ae48 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.082573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 777/1584] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:36 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:37.084088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442377e-c67f-4ed0-b6a6-363a0ca9c47c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.150398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442377e-c67f-4ed0-b6a6-363a0ca9c47c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.156591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 808/1585] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:37 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:37.156591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067d4fd-913c-48da-bb3b-47e60f79ae48 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.158845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75bdfd-49b6-41bb-b81b-ec7437129c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.163565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75bdfd-49b6-41bb-b81b-ec7437129c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.163565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75bdfd-49b6-41bb-b81b-ec7437129c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.163565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75bdfd-49b6-41bb-b81b-ec7437129c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.163565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75bdfd-49b6-41bb-b81b-ec7437129c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.163565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75bdfd-49b6-41bb-b81b-ec7437129c1c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.164855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067d4fd-913c-48da-bb3b-47e60f79ae48 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.170079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f14202-1b4c-45ce-8a9b-e34471e5a026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.170754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f14202-1b4c-45ce-8a9b-e34471e5a026 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.171103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f14202-1b4c-45ce-8a9b-e34471e5a026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.171445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f14202-1b4c-45ce-8a9b-e34471e5a026 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.172149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f14202-1b4c-45ce-8a9b-e34471e5a026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.173990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f14202-1b4c-45ce-8a9b-e34471e5a026 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.181333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 778/1586] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.183290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75bdfd-49b6-41bb-b81b-ec7437129c1c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.185126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 809/1587] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.186340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f14202-1b4c-45ce-8a9b-e34471e5a026 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.192987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:37.196545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f14202-1b4c-45ce-8a9b-e34471e5a026 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.198469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75bdfd-49b6-41bb-b81b-ec7437129c1c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.201281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46f381-53f6-4463-ab6f-3b5fe612d979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.201788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46f381-53f6-4463-ab6f-3b5fe612d979 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.205975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46f381-53f6-4463-ab6f-3b5fe612d979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.205975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46f381-53f6-4463-ab6f-3b5fe612d979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.205975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b46f381-53f6-4463-ab6f-3b5fe612d979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.206386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3dcad-7a98-42d8-acde-4d43d9c347b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.206386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3dcad-7a98-42d8-acde-4d43d9c347b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.206386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3dcad-7a98-42d8-acde-4d43d9c347b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.206641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b46f381-53f6-4463-ab6f-3b5fe612d979 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.206725 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3dcad-7a98-42d8-acde-4d43d9c347b9 admin admin] PATH_INFO: `/v3/projects/f47b0b8077c444beb5876368b9d76363/users/1aa090812e814881af3e130d2615399f/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.215471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:37.216608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 810/1588] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.217888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46f381-53f6-4463-ab6f-3b5fe612d979 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.226737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3dcad-7a98-42d8-acde-4d43d9c347b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47b0b8077c444beb5876368b9d76363, user_id=1aa090812e814881af3e130d2615399f, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.229337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46f381-53f6-4463-ab6f-3b5fe612d979 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.229882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3dcad-7a98-42d8-acde-4d43d9c347b9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.234691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d280e-8ee3-40e5-9c72-c0c6f3646186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.235908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d280e-8ee3-40e5-9c72-c0c6f3646186 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.236359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d280e-8ee3-40e5-9c72-c0c6f3646186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.236469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d280e-8ee3-40e5-9c72-c0c6f3646186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.237564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d280e-8ee3-40e5-9c72-c0c6f3646186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.239104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d280e-8ee3-40e5-9c72-c0c6f3646186 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.250075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 811/1589] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.251036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d280e-8ee3-40e5-9c72-c0c6f3646186 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.261568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d280e-8ee3-40e5-9c72-c0c6f3646186 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.266312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29f311-1ede-4d91-8688-dda7366fa27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.267143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29f311-1ede-4d91-8688-dda7366fa27b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.267427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29f311-1ede-4d91-8688-dda7366fa27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.267748 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29f311-1ede-4d91-8688-dda7366fa27b admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.268335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29f311-1ede-4d91-8688-dda7366fa27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.270020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29f311-1ede-4d91-8688-dda7366fa27b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.280415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 812/1590] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.281264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29f311-1ede-4d91-8688-dda7366fa27b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.294001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29f311-1ede-4d91-8688-dda7366fa27b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.298759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535df25-a755-4400-8078-104ce032703f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.299486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535df25-a755-4400-8078-104ce032703f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.299768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535df25-a755-4400-8078-104ce032703f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.300041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535df25-a755-4400-8078-104ce032703f admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.300656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535df25-a755-4400-8078-104ce032703f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.302624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535df25-a755-4400-8078-104ce032703f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.317563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 779/1591] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:37 2022] PUT /identity/v3/projects/f47b0b8077c444beb5876368b9d76363/users/1aa090812e814881af3e130d2615399f/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:37.320917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3dcad-7a98-42d8-acde-4d43d9c347b9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.382934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 813/1592] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:37 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:37.383282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3dcad-7a98-42d8-acde-4d43d9c347b9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.386754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535df25-a755-4400-8078-104ce032703f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.388632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4cce3-95e9-4fda-8858-245e199f3c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.389240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4cce3-95e9-4fda-8858-245e199f3c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.389548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4cce3-95e9-4fda-8858-245e199f3c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.389824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4cce3-95e9-4fda-8858-245e199f3c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.390467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4cce3-95e9-4fda-8858-245e199f3c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.392768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4cce3-95e9-4fda-8858-245e199f3c02 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.399890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535df25-a755-4400-8078-104ce032703f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.406597 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96351d7d-941f-49e8-9c0c-d1d7541d413e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.407726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96351d7d-941f-49e8-9c0c-d1d7541d413e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.408010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96351d7d-941f-49e8-9c0c-d1d7541d413e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.408104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 780/1593] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.408209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96351d7d-941f-49e8-9c0c-d1d7541d413e admin admin] PATH_INFO: `/v3/projects/d02b8bfcfa134334858ed8b7981348f1/users/b9db0f3de10842eb97dc6fc580f4489a/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.408727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4cce3-95e9-4fda-8858-245e199f3c02 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.420756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4cce3-95e9-4fda-8858-245e199f3c02 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.425664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307917c8-039a-4dfa-8653-a15a73afc988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.426548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307917c8-039a-4dfa-8653-a15a73afc988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.426899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307917c8-039a-4dfa-8653-a15a73afc988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.427248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307917c8-039a-4dfa-8653-a15a73afc988 admin admin] PATH_INFO: `/v3/projects/e443b5bb7bd64e6da3e868f8e7a4208d/users/65001f0ef2ca40989875e08ee912e259/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.433274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96351d7d-941f-49e8-9c0c-d1d7541d413e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d02b8bfcfa134334858ed8b7981348f1, user_id=b9db0f3de10842eb97dc6fc580f4489a, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.435100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-96351d7d-941f-49e8-9c0c-d1d7541d413e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:30:37.435100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 01 05:30:37.435100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: Sep 01 05:30:37.437621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96351d7d-941f-49e8-9c0c-d1d7541d413e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.446128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307917c8-039a-4dfa-8653-a15a73afc988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e443b5bb7bd64e6da3e868f8e7a4208d, user_id=65001f0ef2ca40989875e08ee912e259, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.448907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307917c8-039a-4dfa-8653-a15a73afc988 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.509301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 814/1594] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:37 2022] PUT /identity/v3/projects/d02b8bfcfa134334858ed8b7981348f1/users/b9db0f3de10842eb97dc6fc580f4489a/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:37.510242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96351d7d-941f-49e8-9c0c-d1d7541d413e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.549611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 781/1595] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:37 2022] PUT /identity/v3/projects/e443b5bb7bd64e6da3e868f8e7a4208d/users/65001f0ef2ca40989875e08ee912e259/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:37.549611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307917c8-039a-4dfa-8653-a15a73afc988 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.593064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96351d7d-941f-49e8-9c0c-d1d7541d413e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.631894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307917c8-039a-4dfa-8653-a15a73afc988 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.638829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9132877-80c4-4ca7-bc3c-1f3e70939741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.639722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9132877-80c4-4ca7-bc3c-1f3e70939741 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.640001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9132877-80c4-4ca7-bc3c-1f3e70939741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.640292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9132877-80c4-4ca7-bc3c-1f3e70939741 admin admin] PATH_INFO: `/v3/projects/337817b568b64e98bc2bfd177e984351/users/e9ad5076ef93462fbe614d7fc323935d/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.658656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9132877-80c4-4ca7-bc3c-1f3e70939741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=337817b568b64e98bc2bfd177e984351, user_id=e9ad5076ef93462fbe614d7fc323935d, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.660552 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9543c5f-fc42-4510-877f-b492aaa7e86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.661257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9543c5f-fc42-4510-877f-b492aaa7e86e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.661568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9543c5f-fc42-4510-877f-b492aaa7e86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.661844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9543c5f-fc42-4510-877f-b492aaa7e86e admin admin] PATH_INFO: `/v3/projects/9f237ba116b64751bee9817c19226b3f/users/e529f3610d114981a5a3d37bdf832dab/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.662017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9132877-80c4-4ca7-bc3c-1f3e70939741 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.677654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9543c5f-fc42-4510-877f-b492aaa7e86e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f237ba116b64751bee9817c19226b3f, user_id=e529f3610d114981a5a3d37bdf832dab, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.680329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9543c5f-fc42-4510-877f-b492aaa7e86e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.725971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 782/1596] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:37 2022] PUT /identity/v3/projects/337817b568b64e98bc2bfd177e984351/users/e9ad5076ef93462fbe614d7fc323935d/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:37.725971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9132877-80c4-4ca7-bc3c-1f3e70939741 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.741104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 815/1597] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:37 2022] PUT /identity/v3/projects/9f237ba116b64751bee9817c19226b3f/users/e529f3610d114981a5a3d37bdf832dab/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:37.742924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc4dbd-1489-468d-b41b-a6b774f9851f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.743168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc4dbd-1489-468d-b41b-a6b774f9851f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.743442 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc4dbd-1489-468d-b41b-a6b774f9851f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.775142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1abc4dbd-1489-468d-b41b-a6b774f9851f None None] MFA Rules not processed for user `1aa090812e814881af3e130d2615399f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:37.777858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1abc4dbd-1489-468d-b41b-a6b774f9851f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:37.778326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:37.789370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:37.802664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9132877-80c4-4ca7-bc3c-1f3e70939741 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.850789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 816/1598] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:37.853694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc4dbd-1489-468d-b41b-a6b774f9851f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.878225 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e247e-b06f-44bb-a43c-a916f006dce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.879058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e247e-b06f-44bb-a43c-a916f006dce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.879461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e247e-b06f-44bb-a43c-a916f006dce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.879740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e247e-b06f-44bb-a43c-a916f006dce2 admin admin] PATH_INFO: `/v3/projects/cdbdacacad0f48c6a9238643e67ac6eb/users/fb50b15e44814c6b82b2a75e3c74de0a/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.899638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e247e-b06f-44bb-a43c-a916f006dce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdbdacacad0f48c6a9238643e67ac6eb, user_id=fb50b15e44814c6b82b2a75e3c74de0a, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.902491 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e247e-b06f-44bb-a43c-a916f006dce2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.925070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc4dbd-1489-468d-b41b-a6b774f9851f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.930968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea65ad-13c4-4d43-b014-24703deb157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.931589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea65ad-13c4-4d43-b014-24703deb157d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.931869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea65ad-13c4-4d43-b014-24703deb157d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.932162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea65ad-13c4-4d43-b014-24703deb157d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.932785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea65ad-13c4-4d43-b014-24703deb157d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.934475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea65ad-13c4-4d43-b014-24703deb157d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.946191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 817/1599] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.947298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea65ad-13c4-4d43-b014-24703deb157d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.957760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea65ad-13c4-4d43-b014-24703deb157d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.962736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981bf26-fd30-42d0-a3a3-d5adf16c284e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.963338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981bf26-fd30-42d0-a3a3-d5adf16c284e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.963663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981bf26-fd30-42d0-a3a3-d5adf16c284e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.963930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981bf26-fd30-42d0-a3a3-d5adf16c284e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.964558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981bf26-fd30-42d0-a3a3-d5adf16c284e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.966520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981bf26-fd30-42d0-a3a3-d5adf16c284e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.988001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 818/1600] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.988001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96275c-70f3-47fb-afdc-5376ca316192 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.988001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96275c-70f3-47fb-afdc-5376ca316192 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.988001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96275c-70f3-47fb-afdc-5376ca316192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.988425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 783/1601] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:37 2022] PUT /identity/v3/projects/cdbdacacad0f48c6a9238643e67ac6eb/users/fb50b15e44814c6b82b2a75e3c74de0a/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:37.988425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a81d1d-89b6-4d10-8e09-fdd8ea9394f6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.988425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a81d1d-89b6-4d10-8e09-fdd8ea9394f6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.988425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a81d1d-89b6-4d10-8e09-fdd8ea9394f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.002517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0d96275c-70f3-47fb-afdc-5376ca316192 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:38.004541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0d96275c-70f3-47fb-afdc-5376ca316192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:38.099406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e6a81d1d-89b6-4d10-8e09-fdd8ea9394f6 None None] MFA Rules not processed for user `65001f0ef2ca40989875e08ee912e259`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:38.099406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e6a81d1d-89b6-4d10-8e09-fdd8ea9394f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:38.099892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 819/1602] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:38.099892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d365d6-4200-4b17-991c-99082c743f01 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.099892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d365d6-4200-4b17-991c-99082c743f01 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.099892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d365d6-4200-4b17-991c-99082c743f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.106793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f3d365d6-4200-4b17-991c-99082c743f01 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:38.108028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f3d365d6-4200-4b17-991c-99082c743f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:38.115937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 820/1603] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:38.117157 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b6a6-1417-4098-b9f9-4254f515123e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.117440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b6a6-1417-4098-b9f9-4254f515123e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.117747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b6a6-1417-4098-b9f9-4254f515123e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.121539 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 784/1604] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:38.122723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebabda-bec4-4a77-8245-3eac2e0b35fd None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.123076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebabda-bec4-4a77-8245-3eac2e0b35fd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.123420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebabda-bec4-4a77-8245-3eac2e0b35fd None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.125202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 785/1605] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 1 05:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:30:38.126180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebabda-bec4-4a77-8245-3eac2e0b35fd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:38.147720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebabda-bec4-4a77-8245-3eac2e0b35fd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:38.151247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fb75b6a6-1417-4098-b9f9-4254f515123e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:38.152542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fb75b6a6-1417-4098-b9f9-4254f515123e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:38.155754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b39fb-4792-4ec2-980a-19809ee274fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:38.157623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b39fb-4792-4ec2-980a-19809ee274fd admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.157623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b39fb-4792-4ec2-980a-19809ee274fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.160189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b39fb-4792-4ec2-980a-19809ee274fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.160189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b39fb-4792-4ec2-980a-19809ee274fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:38.160375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b39fb-4792-4ec2-980a-19809ee274fd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:38.161687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 821/1606] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:38.162520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75b6a6-1417-4098-b9f9-4254f515123e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:38.174854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75b6a6-1417-4098-b9f9-4254f515123e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:38.180829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 786/1607] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:38.182188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c0ed-fd9b-4b54-9481-94c29c2260eb None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.182471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c0ed-fd9b-4b54-9481-94c29c2260eb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.182743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c0ed-fd9b-4b54-9481-94c29c2260eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.183701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d802e0-f23e-460e-85dd-92514294ce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:38.184603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d802e0-f23e-460e-85dd-92514294ce34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.184884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d802e0-f23e-460e-85dd-92514294ce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.185160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d802e0-f23e-460e-85dd-92514294ce34 admin admin] PATH_INFO: `/v3/projects/5f2bd01b9c28466682470cbf168ee4ec/users/156b62ff74ba429b88e22c0b47595827/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.202578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d802e0-f23e-460e-85dd-92514294ce34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2bd01b9c28466682470cbf168ee4ec, user_id=156b62ff74ba429b88e22c0b47595827, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:38.205370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3607c0ed-fd9b-4b54-9481-94c29c2260eb None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:38.205631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d802e0-f23e-460e-85dd-92514294ce34 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:38.209332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3607c0ed-fd9b-4b54-9481-94c29c2260eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:38.217058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 787/1608] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:38.217902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c0ed-fd9b-4b54-9481-94c29c2260eb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:38.265890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:38.266307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:38.271279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 822/1609] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:38 2022] PUT /identity/v3/projects/5f2bd01b9c28466682470cbf168ee4ec/users/156b62ff74ba429b88e22c0b47595827/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:38.276047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c0ed-fd9b-4b54-9481-94c29c2260eb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:38.277811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45d37-87f6-4e5f-9c66-81c2e9e44077 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.279020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45d37-87f6-4e5f-9c66-81c2e9e44077 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.280042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45d37-87f6-4e5f-9c66-81c2e9e44077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.320795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-06e45d37-87f6-4e5f-9c66-81c2e9e44077 None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:38.322054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-06e45d37-87f6-4e5f-9c66-81c2e9e44077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:38.330380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03400c9b-e63b-4040-adc5-3e5c17815ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:38.331000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03400c9b-e63b-4040-adc5-3e5c17815ecb service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.331272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03400c9b-e63b-4040-adc5-3e5c17815ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.331797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03400c9b-e63b-4040-adc5-3e5c17815ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.400322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03400c9b-e63b-4040-adc5-3e5c17815ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:38.400968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 823/1610] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 1 05:30:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:38.401904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03400c9b-e63b-4040-adc5-3e5c17815ecb service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:38.402038 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e45d37-87f6-4e5f-9c66-81c2e9e44077 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:38.403329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-06e45d37-87f6-4e5f-9c66-81c2e9e44077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:38.416231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e45d37-87f6-4e5f-9c66-81c2e9e44077 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:38.417796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 788/1611] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:38.419096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03400c9b-e63b-4040-adc5-3e5c17815ecb service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:38.420652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea19a99-bfe9-4e34-8a53-0d86cef1c9c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:38.421279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19a99-bfe9-4e34-8a53-0d86cef1c9c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.422090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19a99-bfe9-4e34-8a53-0d86cef1c9c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.422977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19a99-bfe9-4e34-8a53-0d86cef1c9c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.443881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea19a99-bfe9-4e34-8a53-0d86cef1c9c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:38.445329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea19a99-bfe9-4e34-8a53-0d86cef1c9c8 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:38.456316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 824/1612] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:38.457166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea19a99-bfe9-4e34-8a53-0d86cef1c9c8 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:38.465488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea19a99-bfe9-4e34-8a53-0d86cef1c9c8 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:38.469901 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883fae1c-a919-4d51-abb4-637b7c395cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:38.470501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fae1c-a919-4d51-abb4-637b7c395cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.475815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fae1c-a919-4d51-abb4-637b7c395cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.475815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fae1c-a919-4d51-abb4-637b7c395cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.481713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fae1c-a919-4d51-abb4-637b7c395cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:38.483313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fae1c-a919-4d51-abb4-637b7c395cb1 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:38.489363 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03400c9b-e63b-4040-adc5-3e5c17815ecb service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:38.493794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f28763-feff-4135-bbc0-6ebfca8299da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:38.495151 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f28763-feff-4135-bbc0-6ebfca8299da service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.495446 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f28763-feff-4135-bbc0-6ebfca8299da service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.495720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f28763-feff-4135-bbc0-6ebfca8299da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.497697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 825/1613] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:38.498914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883fae1c-a919-4d51-abb4-637b7c395cb1 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:38.514949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883fae1c-a919-4d51-abb4-637b7c395cb1 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:38.523750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bca69-2490-447a-8a04-dd402cb8cb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:38.524401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bca69-2490-447a-8a04-dd402cb8cb34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.524685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bca69-2490-447a-8a04-dd402cb8cb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.524962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bca69-2490-447a-8a04-dd402cb8cb34 admin admin] PATH_INFO: `/v3/projects/cd8e91d7f43746a792603f48c8b76523/users/6ed92ba22e824b198c5f86ac918dd3cf/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.544470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bca69-2490-447a-8a04-dd402cb8cb34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8e91d7f43746a792603f48c8b76523, user_id=6ed92ba22e824b198c5f86ac918dd3cf, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:38.548070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bca69-2490-447a-8a04-dd402cb8cb34 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:38.602121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f28763-feff-4135-bbc0-6ebfca8299da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:38.606202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f28763-feff-4135-bbc0-6ebfca8299da service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:38.627139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 789/1614] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:38.628070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f28763-feff-4135-bbc0-6ebfca8299da service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:38.651642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:38.677428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 826/1615] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:38 2022] PUT /identity/v3/projects/cd8e91d7f43746a792603f48c8b76523/users/6ed92ba22e824b198c5f86ac918dd3cf/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:38.677801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f28763-feff-4135-bbc0-6ebfca8299da service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:38.677888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cf714-91b6-41e6-ba70-ad06b6cd73dc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.677888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cf714-91b6-41e6-ba70-ad06b6cd73dc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.677888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cf714-91b6-41e6-ba70-ad06b6cd73dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.718632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a13cf714-91b6-41e6-ba70-ad06b6cd73dc None None] MFA Rules not processed for user `156b62ff74ba429b88e22c0b47595827`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:38.722203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a13cf714-91b6-41e6-ba70-ad06b6cd73dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:38.878110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 827/1616] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:38.879198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cf714-91b6-41e6-ba70-ad06b6cd73dc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:38.895071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa65160-888f-4936-b150-10aa2f13f587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:38.898076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa65160-888f-4936-b150-10aa2f13f587 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.898720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa65160-888f-4936-b150-10aa2f13f587 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.898720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa65160-888f-4936-b150-10aa2f13f587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.912855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:39.059941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cf714-91b6-41e6-ba70-ad06b6cd73dc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:39.071420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df2240-1557-4c3a-88ee-72b8afd54c00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:39.072550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df2240-1557-4c3a-88ee-72b8afd54c00 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.072864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa65160-888f-4936-b150-10aa2f13f587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:39.072986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df2240-1557-4c3a-88ee-72b8afd54c00 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.073381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df2240-1557-4c3a-88ee-72b8afd54c00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.075114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa65160-888f-4936-b150-10aa2f13f587 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:39.108496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 790/1617] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:39.112675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90936c-4a39-422c-a718-744a7b76b46a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.120027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90936c-4a39-422c-a718-744a7b76b46a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.120372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90936c-4a39-422c-a718-744a7b76b46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.152970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5a90936c-4a39-422c-a718-744a7b76b46a None None] MFA Rules not processed for user `6ed92ba22e824b198c5f86ac918dd3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:39.155487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5a90936c-4a39-422c-a718-744a7b76b46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:39.163468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df2240-1557-4c3a-88ee-72b8afd54c00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:39.167554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df2240-1557-4c3a-88ee-72b8afd54c00 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:39.182646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 828/1618] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:39.185640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df2240-1557-4c3a-88ee-72b8afd54c00 service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:39.197857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df2240-1557-4c3a-88ee-72b8afd54c00 service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:39.203589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00841ed-2a94-44c2-b638-bd2681095b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:39.204267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00841ed-2a94-44c2-b638-bd2681095b6b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.204551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00841ed-2a94-44c2-b638-bd2681095b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.204805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00841ed-2a94-44c2-b638-bd2681095b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.205670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00841ed-2a94-44c2-b638-bd2681095b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:39.208613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00841ed-2a94-44c2-b638-bd2681095b6b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:39.262393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 791/1619] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:39.298966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757b753-d1a8-4535-b441-016fa2d7057e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.299293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757b753-d1a8-4535-b441-016fa2d7057e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.299587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757b753-d1a8-4535-b441-016fa2d7057e None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.301165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 792/1620] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 1 05:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:30:39.317851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d747b-daf4-42fb-8789-8b7d78f87622 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.318226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d747b-daf4-42fb-8789-8b7d78f87622 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.318440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d747b-daf4-42fb-8789-8b7d78f87622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.339917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 829/1621] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:39 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:39.342951 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00841ed-2a94-44c2-b638-bd2681095b6b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:39.347107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4a5d747b-daf4-42fb-8789-8b7d78f87622 None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:39.348617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4a5d747b-daf4-42fb-8789-8b7d78f87622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:39.407984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:39.503961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 793/1622] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 1 05:30:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:39.505997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d747b-daf4-42fb-8789-8b7d78f87622 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:39.515316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00841ed-2a94-44c2-b638-bd2681095b6b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:39.528801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb1799-fd4b-433b-9372-b03242888c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:39.529446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb1799-fd4b-433b-9372-b03242888c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.529887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb1799-fd4b-433b-9372-b03242888c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.536519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb1799-fd4b-433b-9372-b03242888c32 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.536519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb1799-fd4b-433b-9372-b03242888c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:39.536519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb1799-fd4b-433b-9372-b03242888c32 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:39.544611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d747b-daf4-42fb-8789-8b7d78f87622 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:39.569926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980567a3-098b-4fbb-a389-488086ccf612 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:39.570801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980567a3-098b-4fbb-a389-488086ccf612 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.571310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980567a3-098b-4fbb-a389-488086ccf612 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.571885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980567a3-098b-4fbb-a389-488086ccf612 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.675374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 830/1623] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:39 2022] POST /identity/v3/users => generated 293 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:39.675374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e17f0-a8d5-498b-8ad2-e63fa595ddf2 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.675374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e17f0-a8d5-498b-8ad2-e63fa595ddf2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.675374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e17f0-a8d5-498b-8ad2-e63fa595ddf2 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.681927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 831/1624] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 1 05:30:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:30:39.682932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e17f0-a8d5-498b-8ad2-e63fa595ddf2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:39.694220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980567a3-098b-4fbb-a389-488086ccf612 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:39.697166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980567a3-098b-4fbb-a389-488086ccf612 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:39.705518 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e17f0-a8d5-498b-8ad2-e63fa595ddf2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:39.709621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 794/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:39.710343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e2492-0d2f-40f1-bfe7-264c6b7b6afe None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.710630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e2492-0d2f-40f1-bfe7-264c6b7b6afe None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.710902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e2492-0d2f-40f1-bfe7-264c6b7b6afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.711738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1663d-032d-436f-b1be-335c4676e8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:39.712566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1663d-032d-436f-b1be-335c4676e8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.712846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1663d-032d-436f-b1be-335c4676e8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.713114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1663d-032d-436f-b1be-335c4676e8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.713840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1663d-032d-436f-b1be-335c4676e8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:39.716055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1663d-032d-436f-b1be-335c4676e8e7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:39.737007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f93e2492-0d2f-40f1-bfe7-264c6b7b6afe None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:39.739406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f93e2492-0d2f-40f1-bfe7-264c6b7b6afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:39.740331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 832/1626] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:39.741420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1663d-032d-436f-b1be-335c4676e8e7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:39.762115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1663d-032d-436f-b1be-335c4676e8e7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:39.762235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 795/1627] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:30:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:39.768046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896560b-d06b-442f-b60b-51b8aae5221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:39.769077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896560b-d06b-442f-b60b-51b8aae5221e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.770870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896560b-d06b-442f-b60b-51b8aae5221e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.771313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896560b-d06b-442f-b60b-51b8aae5221e admin admin] PATH_INFO: `/v3/projects/80a5ea90efdf42039daaf1d4835dd79e/users/6f13845e1a984105a4848f953fe20a6b/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.777317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e2492-0d2f-40f1-bfe7-264c6b7b6afe None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:39.793587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:39.834998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896560b-d06b-442f-b60b-51b8aae5221e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a5ea90efdf42039daaf1d4835dd79e, user_id=6f13845e1a984105a4848f953fe20a6b, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:39.841564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896560b-d06b-442f-b60b-51b8aae5221e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:39.898596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e2492-0d2f-40f1-bfe7-264c6b7b6afe None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:39.904062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36163c-fe45-49fb-9877-b7aa902930db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:39.905512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36163c-fe45-49fb-9877-b7aa902930db service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.905772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36163c-fe45-49fb-9877-b7aa902930db service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.906089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36163c-fe45-49fb-9877-b7aa902930db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.921174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36163c-fe45-49fb-9877-b7aa902930db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:39.922893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36163c-fe45-49fb-9877-b7aa902930db service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:39.938355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 796/1628] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:39.959126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36163c-fe45-49fb-9877-b7aa902930db service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:39.960650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 833/1629] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:39 2022] PUT /identity/v3/projects/80a5ea90efdf42039daaf1d4835dd79e/users/6f13845e1a984105a4848f953fe20a6b/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:39.981912 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36163c-fe45-49fb-9877-b7aa902930db service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:40.083902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6fe84-68fb-4213-a61d-e30fb1bb82a7 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.084282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6fe84-68fb-4213-a61d-e30fb1bb82a7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.084563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6fe84-68fb-4213-a61d-e30fb1bb82a7 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:40.086165 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 834/1630] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 1 05:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:30:40.098596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dddbd-f4e7-4fb9-b6f8-7c5eaa611b0e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.098596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dddbd-f4e7-4fb9-b6f8-7c5eaa611b0e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.103133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dddbd-f4e7-4fb9-b6f8-7c5eaa611b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:40.165404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-988dddbd-f4e7-4fb9-b6f8-7c5eaa611b0e None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:40.166669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-988dddbd-f4e7-4fb9-b6f8-7c5eaa611b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:40.177306 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a1284-7f98-469d-a665-d7f4ffc139bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:40.177868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a1284-7f98-469d-a665-d7f4ffc139bf admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.179118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a1284-7f98-469d-a665-d7f4ffc139bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.179118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a1284-7f98-469d-a665-d7f4ffc139bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:40.179803 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a1284-7f98-469d-a665-d7f4ffc139bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:40.191086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a1284-7f98-469d-a665-d7f4ffc139bf admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:40.224379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 797/1631] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:40.229330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a1284-7f98-469d-a665-d7f4ffc139bf admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:40.255451 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a1284-7f98-469d-a665-d7f4ffc139bf admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:40.262937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab44320-75dc-4ea3-8f3a-ba23efe042b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:40.264698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab44320-75dc-4ea3-8f3a-ba23efe042b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.264977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab44320-75dc-4ea3-8f3a-ba23efe042b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.265302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab44320-75dc-4ea3-8f3a-ba23efe042b4 admin admin] PATH_INFO: `/v3/projects/80a5ea90efdf42039daaf1d4835dd79e/users/6f13845e1a984105a4848f953fe20a6b/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:40.269597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab44320-75dc-4ea3-8f3a-ba23efe042b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a5ea90efdf42039daaf1d4835dd79e, user_id=6f13845e1a984105a4848f953fe20a6b, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:40.272952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab44320-75dc-4ea3-8f3a-ba23efe042b4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:40.275994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:40.317254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 798/1632] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:40 2022] PUT /identity/v3/projects/80a5ea90efdf42039daaf1d4835dd79e/users/6f13845e1a984105a4848f953fe20a6b/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:40.317254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab44320-75dc-4ea3-8f3a-ba23efe042b4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:40.344137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 835/1633] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:30:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:40.346217 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb4c6c-c551-4b6d-954d-17df71f4a941 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.347265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb4c6c-c551-4b6d-954d-17df71f4a941 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.349197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb4c6c-c551-4b6d-954d-17df71f4a941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:40.415773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ebbb4c6c-c551-4b6d-954d-17df71f4a941 None None] MFA Rules not processed for user `6f13845e1a984105a4848f953fe20a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:40.416109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ebbb4c6c-c551-4b6d-954d-17df71f4a941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:40.458188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab44320-75dc-4ea3-8f3a-ba23efe042b4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:40.465425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:40.466315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.466592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.466866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] PATH_INFO: `/v3/users/1aa090812e814881af3e130d2615399f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:40.468228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aa090812e814881af3e130d2615399f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:40.470943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:40.535814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 836/1634] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:40.536784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb4c6c-c551-4b6d-954d-17df71f4a941 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:40.627842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb4c6c-c551-4b6d-954d-17df71f4a941 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:40.634504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c782f-5bfc-47df-bb82-4ba0d4702c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:40.635304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c782f-5bfc-47df-bb82-4ba0d4702c33 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.638705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c782f-5bfc-47df-bb82-4ba0d4702c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.638705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c782f-5bfc-47df-bb82-4ba0d4702c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:40.657278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:40.740806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c782f-5bfc-47df-bb82-4ba0d4702c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:40.743051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aa090812e814881af3e130d2615399f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.744210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c782f-5bfc-47df-bb82-4ba0d4702c33 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:40.744685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aa090812e814881af3e130d2615399f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.764472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 837/1635] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:40.767767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c782f-5bfc-47df-bb82-4ba0d4702c33 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:40.779415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c782f-5bfc-47df-bb82-4ba0d4702c33 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:40.792391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f8ab4-242f-45a0-9c23-7fdfeb213f65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:40.793356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f8ab4-242f-45a0-9c23-7fdfeb213f65 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.795266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f8ab4-242f-45a0-9c23-7fdfeb213f65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.795797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f8ab4-242f-45a0-9c23-7fdfeb213f65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:40.798362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8c8f8ab4-242f-45a0-9c23-7fdfeb213f65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:40.840225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f8ab4-242f-45a0-9c23-7fdfeb213f65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:40.843456 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f8ab4-242f-45a0-9c23-7fdfeb213f65 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:40.856648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aa090812e814881af3e130d2615399f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.869805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 838/1636] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:40.883527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aa090812e814881af3e130d2615399f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.900272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aa090812e814881af3e130d2615399f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.920471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 799/1637] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:40 2022] DELETE /identity/v3/users/1aa090812e814881af3e130d2615399f => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:40.927539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711049f-a774-45ba-92fd-470e1393d355 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.927901 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711049f-a774-45ba-92fd-470e1393d355 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.928194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711049f-a774-45ba-92fd-470e1393d355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:41.314969 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b711049f-a774-45ba-92fd-470e1393d355 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:41.314969 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b711049f-a774-45ba-92fd-470e1393d355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:41.314969 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 839/1638] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:41.315965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:41.315965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:41.315965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cabefa-6c23-4af7-83c6-80e4f6207789 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:41.317624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3fa48-7b31-495f-a216-5ee6866a95a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:41.317624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3fa48-7b31-495f-a216-5ee6866a95a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:41.317624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3fa48-7b31-495f-a216-5ee6866a95a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:41.317624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3fa48-7b31-495f-a216-5ee6866a95a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:41.318093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3fa48-7b31-495f-a216-5ee6866a95a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:41.318093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3fa48-7b31-495f-a216-5ee6866a95a7 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:41.318093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 800/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:41.358305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711049f-a774-45ba-92fd-470e1393d355 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:41.360813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b711049f-a774-45ba-92fd-470e1393d355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:41.460505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711049f-a774-45ba-92fd-470e1393d355 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:41.465322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327db2d-c029-449c-b464-50210ea55f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:41.466899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327db2d-c029-449c-b464-50210ea55f52 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:41.467417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327db2d-c029-449c-b464-50210ea55f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:41.467902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327db2d-c029-449c-b464-50210ea55f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:41.473119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3327db2d-c029-449c-b464-50210ea55f52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:41.594934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327db2d-c029-449c-b464-50210ea55f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:41.596868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327db2d-c029-449c-b464-50210ea55f52 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:41.609012 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 840/1640] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:42.011003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:42.612722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3fa48-7b31-495f-a216-5ee6866a95a7 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:42.614203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a6b3fa48-7b31-495f-a216-5ee6866a95a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:42.635829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3fa48-7b31-495f-a216-5ee6866a95a7 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:42.651051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:42.651890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.662111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.662111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] PATH_INFO: `/v3/projects/f47b0b8077c444beb5876368b9d76363` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:42.662111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f47b0b8077c444beb5876368b9d76363)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:42.662111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:30:42.662111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 01 05:30:42.662111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: Sep 01 05:30:42.662111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:42.672937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:42.810902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f47b0b8077c444beb5876368b9d76363', 'request_id': 'req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:42.941473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f47b0b8077c444beb5876368b9d76363', 'request_id': 'req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:42.966533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f47b0b8077c444beb5876368b9d76363', 'request_id': 'req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:42.988346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] The token cache is being invalidate because project f47b0b8077c444beb5876368b9d76363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:42.988346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:42.988346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9dfac3c-3322-4fbd-aeb9-5f2bd8c9a16b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:42.988346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 801/1641] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:42 2022] DELETE /identity/v3/projects/f47b0b8077c444beb5876368b9d76363 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:42.988888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21694e1d-809d-413d-b504-a378a943df6f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.989350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21694e1d-809d-413d-b504-a378a943df6f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.989525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21694e1d-809d-413d-b504-a378a943df6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.015085 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-21694e1d-809d-413d-b504-a378a943df6f None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:43.018420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-21694e1d-809d-413d-b504-a378a943df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:43.145665 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 841/1642] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:43.153795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548045d6-e19c-488d-93b7-2849a963e5ef None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.154988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548045d6-e19c-488d-93b7-2849a963e5ef None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.155598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548045d6-e19c-488d-93b7-2849a963e5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.250676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-548045d6-e19c-488d-93b7-2849a963e5ef None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:43.252212 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-548045d6-e19c-488d-93b7-2849a963e5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:43.268242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21694e1d-809d-413d-b504-a378a943df6f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.270021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 802/1643] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:43.278582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21694e1d-809d-413d-b504-a378a943df6f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:43.285757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f9b79-7bf2-4667-bb26-9ae5293c5ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:43.286550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f9b79-7bf2-4667-bb26-9ae5293c5ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.287117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f9b79-7bf2-4667-bb26-9ae5293c5ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.287532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f9b79-7bf2-4667-bb26-9ae5293c5ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.288326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f9b79-7bf2-4667-bb26-9ae5293c5ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:43.288764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f9b79-7bf2-4667-bb26-9ae5293c5ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:43.291551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f9b79-7bf2-4667-bb26-9ae5293c5ce8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:43.314230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548045d6-e19c-488d-93b7-2849a963e5ef None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.325982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 842/1644] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:43.329392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548045d6-e19c-488d-93b7-2849a963e5ef None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:43.341411 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d930-5bba-42bf-8a90-e5c4073c814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:43.344774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d930-5bba-42bf-8a90-e5c4073c814d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.345161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d930-5bba-42bf-8a90-e5c4073c814d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.345541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d930-5bba-42bf-8a90-e5c4073c814d admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.346321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9d930-5bba-42bf-8a90-e5c4073c814d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:43.353983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9d930-5bba-42bf-8a90-e5c4073c814d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:43.501557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 803/1645] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:43 2022] POST /identity/v3/projects => generated 363 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:43.507158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f9b79-7bf2-4667-bb26-9ae5293c5ce8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.527667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f9b79-7bf2-4667-bb26-9ae5293c5ce8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:43.724573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f530b5-b4c5-4c00-af6f-63ba9f9b7a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:43.726750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f530b5-b4c5-4c00-af6f-63ba9f9b7a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.736943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f530b5-b4c5-4c00-af6f-63ba9f9b7a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.737427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f530b5-b4c5-4c00-af6f-63ba9f9b7a48 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.747599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f530b5-b4c5-4c00-af6f-63ba9f9b7a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:43.751813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f530b5-b4c5-4c00-af6f-63ba9f9b7a48 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:43.967246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 843/1646] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:43 2022] POST /identity/v3/users => generated 282 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:43.967402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d930-5bba-42bf-8a90-e5c4073c814d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.981357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d930-5bba-42bf-8a90-e5c4073c814d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:44.007565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470924b-67f4-4fdf-8d2c-c4305dd9a882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:44.011531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470924b-67f4-4fdf-8d2c-c4305dd9a882 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:44.013080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470924b-67f4-4fdf-8d2c-c4305dd9a882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:44.013379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470924b-67f4-4fdf-8d2c-c4305dd9a882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:44.017328 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470924b-67f4-4fdf-8d2c-c4305dd9a882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:44.022031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470924b-67f4-4fdf-8d2c-c4305dd9a882 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:44.049504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f530b5-b4c5-4c00-af6f-63ba9f9b7a48 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:44.050888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 804/1647] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:44.081529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f530b5-b4c5-4c00-af6f-63ba9f9b7a48 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:44.116173 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163dd9c-b957-4fba-88c6-3180b50f5057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:44.117005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163dd9c-b957-4fba-88c6-3180b50f5057 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:44.119177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163dd9c-b957-4fba-88c6-3180b50f5057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:44.119177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163dd9c-b957-4fba-88c6-3180b50f5057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:44.119177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163dd9c-b957-4fba-88c6-3180b50f5057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:44.127691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163dd9c-b957-4fba-88c6-3180b50f5057 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:44.169212 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470924b-67f4-4fdf-8d2c-c4305dd9a882 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:44.171655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 844/1648] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:44.192308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470924b-67f4-4fdf-8d2c-c4305dd9a882 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:44.209949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc23d6-18bb-4456-af6b-2cdd3695e6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:44.211279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc23d6-18bb-4456-af6b-2cdd3695e6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:44.212420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc23d6-18bb-4456-af6b-2cdd3695e6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:44.214974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc23d6-18bb-4456-af6b-2cdd3695e6b1 admin admin] PATH_INFO: `/v3/projects/b149271f3e5942239d9a7235d544c096/users/80c4fea00a9a4c05b6082c24a7924e26/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:44.268855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc23d6-18bb-4456-af6b-2cdd3695e6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b149271f3e5942239d9a7235d544c096, user_id=80c4fea00a9a4c05b6082c24a7924e26, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:44.276454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc23d6-18bb-4456-af6b-2cdd3695e6b1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:44.414930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 805/1649] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:44 2022] PUT /identity/v3/projects/b149271f3e5942239d9a7235d544c096/users/80c4fea00a9a4c05b6082c24a7924e26/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:44.428891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa480be-5e31-47fd-9904-36e84fa7d68c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:44.429232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa480be-5e31-47fd-9904-36e84fa7d68c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:44.429443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa480be-5e31-47fd-9904-36e84fa7d68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:44.464952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7fa480be-5e31-47fd-9904-36e84fa7d68c None None] MFA Rules not processed for user `80c4fea00a9a4c05b6082c24a7924e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:44.466281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7fa480be-5e31-47fd-9904-36e84fa7d68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:44.526624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:44.572797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 845/1650] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:44.581878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc23d6-18bb-4456-af6b-2cdd3695e6b1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:44.747311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc23d6-18bb-4456-af6b-2cdd3695e6b1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:44.753908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401d2df-0a38-4575-b107-0166e57b567c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:44.754580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401d2df-0a38-4575-b107-0166e57b567c admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:44.754909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401d2df-0a38-4575-b107-0166e57b567c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:44.755334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401d2df-0a38-4575-b107-0166e57b567c admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:44.755985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401d2df-0a38-4575-b107-0166e57b567c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:44.758925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401d2df-0a38-4575-b107-0166e57b567c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:44.910027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 806/1651] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:44 2022] POST /identity/v3/projects => generated 365 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:44.911174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa480be-5e31-47fd-9904-36e84fa7d68c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:44.926577 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa480be-5e31-47fd-9904-36e84fa7d68c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:45.024199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401d2df-0a38-4575-b107-0166e57b567c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:45.026231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6401d2df-0a38-4575-b107-0166e57b567c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:45.084493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:45.099065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf3d8b-6867-4b50-bc12-2776f85c1712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:45.101604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf3d8b-6867-4b50-bc12-2776f85c1712 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:45.101944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf3d8b-6867-4b50-bc12-2776f85c1712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:45.103413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf3d8b-6867-4b50-bc12-2776f85c1712 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:45.103413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf3d8b-6867-4b50-bc12-2776f85c1712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:45.110198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf3d8b-6867-4b50-bc12-2776f85c1712 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:45.248898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401d2df-0a38-4575-b107-0166e57b567c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:45.261700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 846/1652] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:44 2022] POST /identity/v3/users => generated 289 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:45.263213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf3d8b-6867-4b50-bc12-2776f85c1712 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:45.265490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:45.267817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:45.268770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:45.270055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] PATH_INFO: `/v3/users/6ed92ba22e824b198c5f86ac918dd3cf` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:45.274339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ed92ba22e824b198c5f86ac918dd3cf)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:45.282136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf3d8b-6867-4b50-bc12-2776f85c1712 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:45.283529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:45.302755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899df2b4-9924-4684-8a71-9d54bfcde194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:45.303406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899df2b4-9924-4684-8a71-9d54bfcde194 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:45.303691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899df2b4-9924-4684-8a71-9d54bfcde194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:45.303965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899df2b4-9924-4684-8a71-9d54bfcde194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:45.304631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899df2b4-9924-4684-8a71-9d54bfcde194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:45.306309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899df2b4-9924-4684-8a71-9d54bfcde194 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:45.337639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 847/1653] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:45.342599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899df2b4-9924-4684-8a71-9d54bfcde194 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:45.362328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899df2b4-9924-4684-8a71-9d54bfcde194 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:45.391557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8753c8-9fa5-4d32-83a6-d78cf8f40a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:45.392158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8753c8-9fa5-4d32-83a6-d78cf8f40a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:45.392464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8753c8-9fa5-4d32-83a6-d78cf8f40a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:45.392771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8753c8-9fa5-4d32-83a6-d78cf8f40a37 admin admin] PATH_INFO: `/v3/projects/98820d18dabd4e3f957ac01b6f37d348/users/a9ae367f320f4289aa4bbe61f9003f77/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:45.495012 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8753c8-9fa5-4d32-83a6-d78cf8f40a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98820d18dabd4e3f957ac01b6f37d348, user_id=a9ae367f320f4289aa4bbe61f9003f77, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:45.495012 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8753c8-9fa5-4d32-83a6-d78cf8f40a37 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:46.245948 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 848/1654] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:45 2022] PUT /identity/v3/projects/98820d18dabd4e3f957ac01b6f37d348/users/a9ae367f320f4289aa4bbe61f9003f77/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 904 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:46.249645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8753c8-9fa5-4d32-83a6-d78cf8f40a37 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:46.385661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ed92ba22e824b198c5f86ac918dd3cf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:46.388044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ed92ba22e824b198c5f86ac918dd3cf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:46.462872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8753c8-9fa5-4d32-83a6-d78cf8f40a37 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:46.488686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070d843-9d0b-4544-a637-42d2715271f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:46.489325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070d843-9d0b-4544-a637-42d2715271f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:46.489638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070d843-9d0b-4544-a637-42d2715271f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:46.489918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070d843-9d0b-4544-a637-42d2715271f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:46.490576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070d843-9d0b-4544-a637-42d2715271f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:46.493939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ed92ba22e824b198c5f86ac918dd3cf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:46.495983 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070d843-9d0b-4544-a637-42d2715271f2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:46.531231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 849/1655] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:46.536145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ed92ba22e824b198c5f86ac918dd3cf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:46.536997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070d843-9d0b-4544-a637-42d2715271f2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:46.543975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d070d843-9d0b-4544-a637-42d2715271f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:46.569493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ba7da3dd-3e70-4830-9770-8dea160e9d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ed92ba22e824b198c5f86ac918dd3cf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:46.591829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070d843-9d0b-4544-a637-42d2715271f2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:46.595461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 807/1656] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:45 2022] DELETE /identity/v3/users/6ed92ba22e824b198c5f86ac918dd3cf => generated 0 bytes in 1569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:46.597147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887a312-bf46-43da-95da-42998d4c3a06 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:46.598449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887a312-bf46-43da-95da-42998d4c3a06 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:46.602146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887a312-bf46-43da-95da-42998d4c3a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:46.635930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3887a312-bf46-43da-95da-42998d4c3a06 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:46.637969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3887a312-bf46-43da-95da-42998d4c3a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:46.745099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:46.783710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a90f93-8a33-48d5-8ab1-e0025faf2b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:46.787027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a90f93-8a33-48d5-8ab1-e0025faf2b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:46.787027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a90f93-8a33-48d5-8ab1-e0025faf2b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:46.787027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a90f93-8a33-48d5-8ab1-e0025faf2b13 admin admin] PATH_INFO: `/v3/projects/98820d18dabd4e3f957ac01b6f37d348/users/a9ae367f320f4289aa4bbe61f9003f77/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:46.794711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a90f93-8a33-48d5-8ab1-e0025faf2b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98820d18dabd4e3f957ac01b6f37d348, user_id=a9ae367f320f4289aa4bbe61f9003f77, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:46.801871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a90f93-8a33-48d5-8ab1-e0025faf2b13 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:46.821634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 808/1657] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:46.833249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 850/1658] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:46 2022] PUT /identity/v3/projects/98820d18dabd4e3f957ac01b6f37d348/users/a9ae367f320f4289aa4bbe61f9003f77/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:46.844873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887a312-bf46-43da-95da-42998d4c3a06 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:46.868523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3887a312-bf46-43da-95da-42998d4c3a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:46.868614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f84a48-88d8-4856-a625-fe975235a139 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:46.868614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f84a48-88d8-4856-a625-fe975235a139 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:46.868614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f84a48-88d8-4856-a625-fe975235a139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:46.961569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-18f84a48-88d8-4856-a625-fe975235a139 None None] MFA Rules not processed for user `a9ae367f320f4289aa4bbe61f9003f77`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:46.966956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-18f84a48-88d8-4856-a625-fe975235a139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:47.002561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887a312-bf46-43da-95da-42998d4c3a06 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:47.101839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 851/1659] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:47.112382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa46cb3-b2df-4382-8f32-256672c84984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:47.113183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa46cb3-b2df-4382-8f32-256672c84984 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:47.113708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa46cb3-b2df-4382-8f32-256672c84984 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:47.114133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa46cb3-b2df-4382-8f32-256672c84984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:47.117638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f84a48-88d8-4856-a625-fe975235a139 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:47.119599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-18f84a48-88d8-4856-a625-fe975235a139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:47.251316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa46cb3-b2df-4382-8f32-256672c84984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:47.255553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa46cb3-b2df-4382-8f32-256672c84984 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:47.273272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f84a48-88d8-4856-a625-fe975235a139 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:47.279814 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 809/1660] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:47.290589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d8028-6dff-4f46-aa93-305a97def6d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:47.294417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d8028-6dff-4f46-aa93-305a97def6d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:47.294417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d8028-6dff-4f46-aa93-305a97def6d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:47.294417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d8028-6dff-4f46-aa93-305a97def6d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:47.334356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d8028-6dff-4f46-aa93-305a97def6d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:47.337805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d8028-6dff-4f46-aa93-305a97def6d7 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:47.379789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 852/1661] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:47.882235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:48.745095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa46cb3-b2df-4382-8f32-256672c84984 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:48.758373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa46cb3-b2df-4382-8f32-256672c84984 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:48.773848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:48.774680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:48.774952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:48.775435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] PATH_INFO: `/v3/users/80c4fea00a9a4c05b6082c24a7924e26` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:48.776973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c4fea00a9a4c05b6082c24a7924e26)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:48.781730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.040755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c4fea00a9a4c05b6082c24a7924e26'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:49.042730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c4fea00a9a4c05b6082c24a7924e26'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:49.174472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c4fea00a9a4c05b6082c24a7924e26'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:49.202433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c4fea00a9a4c05b6082c24a7924e26'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:49.226018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c4fea00a9a4c05b6082c24a7924e26'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:49.251856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 810/1662] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:48 2022] DELETE /identity/v3/users/80c4fea00a9a4c05b6082c24a7924e26 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:49.255054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118065b8-d4d2-4016-9208-e4dff7383b0b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.255054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118065b8-d4d2-4016-9208-e4dff7383b0b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.255480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118065b8-d4d2-4016-9208-e4dff7383b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.293106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-118065b8-d4d2-4016-9208-e4dff7383b0b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:49.308048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-118065b8-d4d2-4016-9208-e4dff7383b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:49.419713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:49.451987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 853/1663] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:49.495790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.497059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:49.622616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de48014-94d1-429a-a296-60c597998f7c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.643958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962df19-1900-43b2-b82a-c04274def2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.644450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962df19-1900-43b2-b82a-c04274def2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.644730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962df19-1900-43b2-b82a-c04274def2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.645003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962df19-1900-43b2-b82a-c04274def2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.662773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962df19-1900-43b2-b82a-c04274def2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.664246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962df19-1900-43b2-b82a-c04274def2c9 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.695893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 811/1664] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:49.722431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:50.033054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118065b8-d4d2-4016-9208-e4dff7383b0b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.035139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-118065b8-d4d2-4016-9208-e4dff7383b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.062951 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118065b8-d4d2-4016-9208-e4dff7383b0b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.077597 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.079068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.079460 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.079736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] PATH_INFO: `/v3/projects/cd8e91d7f43746a792603f48c8b76523` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.085563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8e91d7f43746a792603f48c8b76523)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.089750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.275624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8e91d7f43746a792603f48c8b76523', 'request_id': 'req-841a56da-d279-4b3f-ab50-231eb041f994'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:50.351126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8e91d7f43746a792603f48c8b76523', 'request_id': 'req-841a56da-d279-4b3f-ab50-231eb041f994'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:50.360320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8e91d7f43746a792603f48c8b76523', 'request_id': 'req-841a56da-d279-4b3f-ab50-231eb041f994'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:50.390729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] The token cache is being invalidate because project cd8e91d7f43746a792603f48c8b76523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:50.391558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:50.392472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-841a56da-d279-4b3f-ab50-231eb041f994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:50.400776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 854/1665] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:50 2022] DELETE /identity/v3/projects/cd8e91d7f43746a792603f48c8b76523 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:50.405358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb1744-0080-4660-8927-0e5d0270a4cb None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.409637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb1744-0080-4660-8927-0e5d0270a4cb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.409637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb1744-0080-4660-8927-0e5d0270a4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.466680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c5fb1744-0080-4660-8927-0e5d0270a4cb None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:50.467932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c5fb1744-0080-4660-8927-0e5d0270a4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.592403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 812/1666] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:50.600681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfdf9a-9d13-4bab-b87e-44ac50804b04 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.601048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfdf9a-9d13-4bab-b87e-44ac50804b04 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.601433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfdf9a-9d13-4bab-b87e-44ac50804b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.651215 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4adfdf9a-9d13-4bab-b87e-44ac50804b04 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:50.653037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4adfdf9a-9d13-4bab-b87e-44ac50804b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.712748 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 855/1667] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:50.713266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb1744-0080-4660-8927-0e5d0270a4cb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.715054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb1744-0080-4660-8927-0e5d0270a4cb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.736082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe2081-e31c-46a3-aee9-78ea93372eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.736957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe2081-e31c-46a3-aee9-78ea93372eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.738038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe2081-e31c-46a3-aee9-78ea93372eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.742139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe2081-e31c-46a3-aee9-78ea93372eb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.743047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe2081-e31c-46a3-aee9-78ea93372eb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:50.745881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe2081-e31c-46a3-aee9-78ea93372eb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.751390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe2081-e31c-46a3-aee9-78ea93372eb4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.788834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 813/1668] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:50.793051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfdf9a-9d13-4bab-b87e-44ac50804b04 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.819664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfdf9a-9d13-4bab-b87e-44ac50804b04 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.840707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a851c9-2865-44c7-9a76-95aff6bc5857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.841343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a851c9-2865-44c7-9a76-95aff6bc5857 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.841731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a851c9-2865-44c7-9a76-95aff6bc5857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.842019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a851c9-2865-44c7-9a76-95aff6bc5857 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.849049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a851c9-2865-44c7-9a76-95aff6bc5857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.850228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a851c9-2865-44c7-9a76-95aff6bc5857 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.043654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 856/1669] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:50 2022] POST /identity/v3/projects => generated 373 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:51.048578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe2081-e31c-46a3-aee9-78ea93372eb4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.067094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe2081-e31c-46a3-aee9-78ea93372eb4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.110387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182cf38-a7df-4bdf-941d-9905ebe4e6bd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.114557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182cf38-a7df-4bdf-941d-9905ebe4e6bd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.114557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182cf38-a7df-4bdf-941d-9905ebe4e6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.239164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6182cf38-a7df-4bdf-941d-9905ebe4e6bd None None] MFA Rules not processed for user `e9ad5076ef93462fbe614d7fc323935d`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:51.246968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6182cf38-a7df-4bdf-941d-9905ebe4e6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:51.383650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1c7d4-7490-4d64-9ee6-847281f8a110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.384295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1c7d4-7490-4d64-9ee6-847281f8a110 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.384582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1c7d4-7490-4d64-9ee6-847281f8a110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.384860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1c7d4-7490-4d64-9ee6-847281f8a110 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.389003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1c7d4-7490-4d64-9ee6-847281f8a110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.391192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1c7d4-7490-4d64-9ee6-847281f8a110 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.513434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 857/1670] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:51.532408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182cf38-a7df-4bdf-941d-9905ebe4e6bd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.534711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6182cf38-a7df-4bdf-941d-9905ebe4e6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:51.557811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 814/1671] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:51 2022] POST /identity/v3/users => generated 287 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:51.559664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1c7d4-7490-4d64-9ee6-847281f8a110 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.579055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1c7d4-7490-4d64-9ee6-847281f8a110 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.593312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5702f82-be12-48a9-8aca-509fc2938fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.594444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5702f82-be12-48a9-8aca-509fc2938fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.596959 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5702f82-be12-48a9-8aca-509fc2938fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.598322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5702f82-be12-48a9-8aca-509fc2938fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.602165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5702f82-be12-48a9-8aca-509fc2938fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.604081 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5702f82-be12-48a9-8aca-509fc2938fa7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.635997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:51.648791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 815/1672] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:51.650279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5702f82-be12-48a9-8aca-509fc2938fa7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.679578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5702f82-be12-48a9-8aca-509fc2938fa7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.691100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182cf38-a7df-4bdf-941d-9905ebe4e6bd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.696870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cec25-bc93-49fe-9bbd-cc55916eb002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.697959 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cec25-bc93-49fe-9bbd-cc55916eb002 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.698388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cec25-bc93-49fe-9bbd-cc55916eb002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.698822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cec25-bc93-49fe-9bbd-cc55916eb002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.699888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cec25-bc93-49fe-9bbd-cc55916eb002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.701749 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cec25-bc93-49fe-9bbd-cc55916eb002 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.720869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8213d71-e1ed-4d74-a350-6325e9366b91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.721544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8213d71-e1ed-4d74-a350-6325e9366b91 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.721832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8213d71-e1ed-4d74-a350-6325e9366b91 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.722106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8213d71-e1ed-4d74-a350-6325e9366b91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.742870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 816/1673] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:51.750363 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cec25-bc93-49fe-9bbd-cc55916eb002 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.762166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8213d71-e1ed-4d74-a350-6325e9366b91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.770544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8213d71-e1ed-4d74-a350-6325e9366b91 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.794156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cec25-bc93-49fe-9bbd-cc55916eb002 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.809189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 858/1674] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:51.812856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba1d7b-8471-4391-a774-6d664c3174ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.816925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba1d7b-8471-4391-a774-6d664c3174ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.817222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba1d7b-8471-4391-a774-6d664c3174ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.817544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba1d7b-8471-4391-a774-6d664c3174ee admin admin] PATH_INFO: `/v3/projects/796add2986054d78ad461b63a33d9e09/users/eaa530c977ab45039acb696a6a1d214e/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.894389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:51.905014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba1d7b-8471-4391-a774-6d664c3174ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=796add2986054d78ad461b63a33d9e09, user_id=eaa530c977ab45039acb696a6a1d214e, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.922379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba1d7b-8471-4391-a774-6d664c3174ee admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.039170 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 817/1675] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:51 2022] PUT /identity/v3/projects/796add2986054d78ad461b63a33d9e09/users/eaa530c977ab45039acb696a6a1d214e/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:52.051361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ebd62-a87c-41a0-987e-77279feaefb3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.051361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ebd62-a87c-41a0-987e-77279feaefb3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.053111 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ebd62-a87c-41a0-987e-77279feaefb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.095844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba1d7b-8471-4391-a774-6d664c3174ee admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.108944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-44ba1d7b-8471-4391-a774-6d664c3174ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:52.158973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4e9ebd62-a87c-41a0-987e-77279feaefb3 None None] MFA Rules not processed for user `eaa530c977ab45039acb696a6a1d214e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:52.158973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4e9ebd62-a87c-41a0-987e-77279feaefb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:52.391407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 859/1676] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:52.396447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ebd62-a87c-41a0-987e-77279feaefb3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.425560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba1d7b-8471-4391-a774-6d664c3174ee admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:52.432901 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ebd62-a87c-41a0-987e-77279feaefb3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:52.436860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:52.438850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.439293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.439716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] PATH_INFO: `/v3/projects/b149271f3e5942239d9a7235d544c096` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.446175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b149271f3e5942239d9a7235d544c096)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:52.446175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.446353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfe90c-6c34-4373-8bb1-2d2362ecaf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:52.446353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfe90c-6c34-4373-8bb1-2d2362ecaf88 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.446353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfe90c-6c34-4373-8bb1-2d2362ecaf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.446353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfe90c-6c34-4373-8bb1-2d2362ecaf88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.446821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfe90c-6c34-4373-8bb1-2d2362ecaf88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:52.448803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfe90c-6c34-4373-8bb1-2d2362ecaf88 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.573024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 860/1677] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:52 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:52.576933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfe90c-6c34-4373-8bb1-2d2362ecaf88 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.679762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b149271f3e5942239d9a7235d544c096', 'request_id': 'req-0e49a7d1-fb75-4522-8ef4-20340a7c9042'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:52.774033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b149271f3e5942239d9a7235d544c096', 'request_id': 'req-0e49a7d1-fb75-4522-8ef4-20340a7c9042'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:52.788947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b149271f3e5942239d9a7235d544c096', 'request_id': 'req-0e49a7d1-fb75-4522-8ef4-20340a7c9042'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:52.846838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] The token cache is being invalidate because project b149271f3e5942239d9a7235d544c096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:52.851720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:52.852461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:52.864368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 818/1678] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:52 2022] DELETE /identity/v3/projects/b149271f3e5942239d9a7235d544c096 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:52.866449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.886383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfe90c-6c34-4373-8bb1-2d2362ecaf88 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:52.894102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:53.011880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:53.060538 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49a7d1-fb75-4522-8ef4-20340a7c9042 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:53.080328 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:53.083643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.084345 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.084626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] PATH_INFO: `/v3/users/a9ae367f320f4289aa4bbe61f9003f77` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.086202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ae367f320f4289aa4bbe61f9003f77)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:53.087818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b6722-ee21-4ec1-9b88-f36e8520884e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:53.088606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b6722-ee21-4ec1-9b88-f36e8520884e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.090339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b6722-ee21-4ec1-9b88-f36e8520884e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.090691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b6722-ee21-4ec1-9b88-f36e8520884e admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.091955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:53.098140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b6722-ee21-4ec1-9b88-f36e8520884e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:53.101688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b6722-ee21-4ec1-9b88-f36e8520884e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:53.123524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:53.221978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 861/1679] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:52 2022] POST /identity/v3/users => generated 292 bytes in 645 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:53.228997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4538e2e-8dc3-40d5-9354-2e138b1b33a1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.228997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4538e2e-8dc3-40d5-9354-2e138b1b33a1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.228997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4538e2e-8dc3-40d5-9354-2e138b1b33a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.271458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a4538e2e-8dc3-40d5-9354-2e138b1b33a1 None None] MFA Rules not processed for user `b9db0f3de10842eb97dc6fc580f4489a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:53.272929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a4538e2e-8dc3-40d5-9354-2e138b1b33a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:53.362399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ae367f320f4289aa4bbe61f9003f77'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:53.363586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ae367f320f4289aa4bbe61f9003f77'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:53.409913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 862/1680] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:53.411010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4538e2e-8dc3-40d5-9354-2e138b1b33a1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:53.412506 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a4538e2e-8dc3-40d5-9354-2e138b1b33a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:53.428624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4538e2e-8dc3-40d5-9354-2e138b1b33a1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:53.434662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feea91f-4039-4e3f-92b0-696e94d3ab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:53.435647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feea91f-4039-4e3f-92b0-696e94d3ab16 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.435973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feea91f-4039-4e3f-92b0-696e94d3ab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.436254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feea91f-4039-4e3f-92b0-696e94d3ab16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.437008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feea91f-4039-4e3f-92b0-696e94d3ab16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:53.439938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feea91f-4039-4e3f-92b0-696e94d3ab16 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:53.457350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 863/1681] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:53.460267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feea91f-4039-4e3f-92b0-696e94d3ab16 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:53.463274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0feea91f-4039-4e3f-92b0-696e94d3ab16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:53.478534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ae367f320f4289aa4bbe61f9003f77'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:53.501392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ae367f320f4289aa4bbe61f9003f77'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:53.530367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ae367f320f4289aa4bbe61f9003f77'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:53.546795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 819/1682] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:52 2022] DELETE /identity/v3/users/a9ae367f320f4289aa4bbe61f9003f77 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:53.548217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:53.594075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feea91f-4039-4e3f-92b0-696e94d3ab16 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:53.700483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1bf3a-6df4-40fa-8784-5e5eb7e14c06 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:53.710016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b571b-0131-42cb-adce-e9626da1b9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:53.718963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b571b-0131-42cb-adce-e9626da1b9b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.718963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b571b-0131-42cb-adce-e9626da1b9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.718963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b571b-0131-42cb-adce-e9626da1b9b8 admin admin] PATH_INFO: `/v3/projects/60e47bd23f144969afd1d335826c9fda/users/738e5799aa16425d81969bcc914e2a55/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.755313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b571b-0131-42cb-adce-e9626da1b9b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e47bd23f144969afd1d335826c9fda, user_id=738e5799aa16425d81969bcc914e2a55, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:53.764465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b571b-0131-42cb-adce-e9626da1b9b8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:53.806289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdadfed-d8d3-4547-8d99-a2b9d453118f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:53.806932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdadfed-d8d3-4547-8d99-a2b9d453118f service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.807267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdadfed-d8d3-4547-8d99-a2b9d453118f service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.807543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdadfed-d8d3-4547-8d99-a2b9d453118f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.828448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:53.875646 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 820/1683] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:53 2022] PUT /identity/v3/projects/60e47bd23f144969afd1d335826c9fda/users/738e5799aa16425d81969bcc914e2a55/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:53.878043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b571b-0131-42cb-adce-e9626da1b9b8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:53.978305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdadfed-d8d3-4547-8d99-a2b9d453118f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:53.983581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdadfed-d8d3-4547-8d99-a2b9d453118f service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:54.041753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 864/1684] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:54.045335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b571b-0131-42cb-adce-e9626da1b9b8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:54.074445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562712f5-24c1-4f1f-856a-a2086f9aa2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:54.075275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562712f5-24c1-4f1f-856a-a2086f9aa2ab admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:54.075793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562712f5-24c1-4f1f-856a-a2086f9aa2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:54.076199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562712f5-24c1-4f1f-856a-a2086f9aa2ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:54.076965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562712f5-24c1-4f1f-856a-a2086f9aa2ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:54.079327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562712f5-24c1-4f1f-856a-a2086f9aa2ab admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:54.103027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdadfed-d8d3-4547-8d99-a2b9d453118f service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:54.104779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 821/1685] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:54.125631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdadfed-d8d3-4547-8d99-a2b9d453118f service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:54.154595 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad69bd2-01a1-4f5d-adb5-9bc69ba36230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:54.155300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad69bd2-01a1-4f5d-adb5-9bc69ba36230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:54.156462 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad69bd2-01a1-4f5d-adb5-9bc69ba36230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:54.156879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad69bd2-01a1-4f5d-adb5-9bc69ba36230 admin admin] PATH_INFO: `/v3/projects/60e47bd23f144969afd1d335826c9fda/users/738e5799aa16425d81969bcc914e2a55/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:54.174957 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad69bd2-01a1-4f5d-adb5-9bc69ba36230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e47bd23f144969afd1d335826c9fda, user_id=738e5799aa16425d81969bcc914e2a55, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:54.177340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad69bd2-01a1-4f5d-adb5-9bc69ba36230 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:54.216728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 865/1686] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:54 2022] PUT /identity/v3/projects/60e47bd23f144969afd1d335826c9fda/users/738e5799aa16425d81969bcc914e2a55/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:54.227869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba96f5-a858-4e15-94df-25f573b50b37 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:54.233397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba96f5-a858-4e15-94df-25f573b50b37 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:54.233397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba96f5-a858-4e15-94df-25f573b50b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:54.307483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c5ba96f5-a858-4e15-94df-25f573b50b37 None None] MFA Rules not processed for user `738e5799aa16425d81969bcc914e2a55`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:54.309662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c5ba96f5-a858-4e15-94df-25f573b50b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:54.393832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:54.447581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 822/1687] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:54.455304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad69bd2-01a1-4f5d-adb5-9bc69ba36230 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:54.457079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7ad69bd2-01a1-4f5d-adb5-9bc69ba36230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:54.561022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad69bd2-01a1-4f5d-adb5-9bc69ba36230 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:54.569025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3eb0a-b9f7-4013-b5ed-53933b56af0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:54.569793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3eb0a-b9f7-4013-b5ed-53933b56af0a service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:54.570147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3eb0a-b9f7-4013-b5ed-53933b56af0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:54.570499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3eb0a-b9f7-4013-b5ed-53933b56af0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:54.585328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3eb0a-b9f7-4013-b5ed-53933b56af0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:54.587277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3eb0a-b9f7-4013-b5ed-53933b56af0a service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:54.602504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 866/1688] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:54.653056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba96f5-a858-4e15-94df-25f573b50b37 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:54.653056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c5ba96f5-a858-4e15-94df-25f573b50b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:54.748907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba96f5-a858-4e15-94df-25f573b50b37 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:54.763410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b71bd-236c-42d5-bd49-6c4bdfbabef2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:54.764016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b71bd-236c-42d5-bd49-6c4bdfbabef2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:54.764421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b71bd-236c-42d5-bd49-6c4bdfbabef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:54.773406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:54.778237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:54.778237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:54.778237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] PATH_INFO: `/v3/projects/98820d18dabd4e3f957ac01b6f37d348` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:54.786816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98820d18dabd4e3f957ac01b6f37d348)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:54.793398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:54.832591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-084b71bd-236c-42d5-bd49-6c4bdfbabef2 None None] MFA Rules not processed for user `e529f3610d114981a5a3d37bdf832dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:54.834724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-084b71bd-236c-42d5-bd49-6c4bdfbabef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:54.967559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98820d18dabd4e3f957ac01b6f37d348', 'request_id': 'req-36691f1c-3524-4c86-9d35-f9536977a647'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:54.975014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:54.980842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 867/1689] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:54.991900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12a172-2542-4697-9766-23f94104563d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:54.992303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12a172-2542-4697-9766-23f94104563d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:54.996743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12a172-2542-4697-9766-23f94104563d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:55.035443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1d12a172-2542-4697-9766-23f94104563d None None] MFA Rules not processed for user `e529f3610d114981a5a3d37bdf832dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:55.044965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98820d18dabd4e3f957ac01b6f37d348', 'request_id': 'req-36691f1c-3524-4c86-9d35-f9536977a647'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:55.058822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98820d18dabd4e3f957ac01b6f37d348', 'request_id': 'req-36691f1c-3524-4c86-9d35-f9536977a647'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:55.060645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1d12a172-2542-4697-9766-23f94104563d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:55.090522 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] The token cache is being invalidate because project 98820d18dabd4e3f957ac01b6f37d348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:55.099525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:55.099525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-36691f1c-3524-4c86-9d35-f9536977a647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:55.100597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 823/1690] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:54 2022] DELETE /identity/v3/projects/98820d18dabd4e3f957ac01b6f37d348 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:55.104980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9cfc-6bf7-4687-b621-a6a2d9651abd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:55.105264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9cfc-6bf7-4687-b621-a6a2d9651abd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:55.105834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9cfc-6bf7-4687-b621-a6a2d9651abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:55.146906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a42e9cfc-6bf7-4687-b621-a6a2d9651abd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:55.148129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a42e9cfc-6bf7-4687-b621-a6a2d9651abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:55.210485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 868/1691] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:55.224439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12a172-2542-4697-9766-23f94104563d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:55.229844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1d12a172-2542-4697-9766-23f94104563d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:55.283699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 824/1692] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:55.292440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ee971-31e1-4a9c-b3ae-50cc1fe3b904 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:55.292776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ee971-31e1-4a9c-b3ae-50cc1fe3b904 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:55.293092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ee971-31e1-4a9c-b3ae-50cc1fe3b904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:55.319202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1a9ee971-31e1-4a9c-b3ae-50cc1fe3b904 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:55.320609 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1a9ee971-31e1-4a9c-b3ae-50cc1fe3b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:55.338603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 825/1693] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:55.345020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ee971-31e1-4a9c-b3ae-50cc1fe3b904 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:55.365777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ee971-31e1-4a9c-b3ae-50cc1fe3b904 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:55.368133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12a172-2542-4697-9766-23f94104563d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:55.371388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22841091-e350-4ab4-aba6-5812b4b70bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:55.372302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22841091-e350-4ab4-aba6-5812b4b70bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:55.373012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22841091-e350-4ab4-aba6-5812b4b70bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:55.373503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22841091-e350-4ab4-aba6-5812b4b70bf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:55.375937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22841091-e350-4ab4-aba6-5812b4b70bf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:55.376358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22841091-e350-4ab4-aba6-5812b4b70bf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:55.379329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22841091-e350-4ab4-aba6-5812b4b70bf5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:55.402685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a328f4-268c-48fb-b740-1948293a86cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:55.404830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a328f4-268c-48fb-b740-1948293a86cd service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:55.408066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 826/1694] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:55.410094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a328f4-268c-48fb-b740-1948293a86cd service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:55.410441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a328f4-268c-48fb-b740-1948293a86cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:55.413113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22841091-e350-4ab4-aba6-5812b4b70bf5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:55.413354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b2a328f4-268c-48fb-b740-1948293a86cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:55.430431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a328f4-268c-48fb-b740-1948293a86cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:55.432055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a328f4-268c-48fb-b740-1948293a86cd service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:55.434563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22841091-e350-4ab4-aba6-5812b4b70bf5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:55.449230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 869/1695] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:55.449753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad254d7-af08-48af-997c-610de5251e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:55.450584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad254d7-af08-48af-997c-610de5251e38 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:55.450687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad254d7-af08-48af-997c-610de5251e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:55.450967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad254d7-af08-48af-997c-610de5251e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:55.452233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad254d7-af08-48af-997c-610de5251e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:55.455538 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad254d7-af08-48af-997c-610de5251e38 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:55.583680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 827/1696] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:55 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:55.587854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a328f4-268c-48fb-b740-1948293a86cd service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:55.603840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a328f4-268c-48fb-b740-1948293a86cd service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:55.688914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9bb98-147d-4382-8ff6-898a12406341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:55.689193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9bb98-147d-4382-8ff6-898a12406341 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:55.690937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9bb98-147d-4382-8ff6-898a12406341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:55.690937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9bb98-147d-4382-8ff6-898a12406341 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:55.690937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9bb98-147d-4382-8ff6-898a12406341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:55.699981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9bb98-147d-4382-8ff6-898a12406341 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:55.946123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 870/1697] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:55 2022] POST /identity/v3/users => generated 289 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:55.947313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad254d7-af08-48af-997c-610de5251e38 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:55.963733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad254d7-af08-48af-997c-610de5251e38 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:55.989761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea8c86-ddc8-4574-bcc1-2f8a968b6f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:55.990383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea8c86-ddc8-4574-bcc1-2f8a968b6f75 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:55.990586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea8c86-ddc8-4574-bcc1-2f8a968b6f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:55.990863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea8c86-ddc8-4574-bcc1-2f8a968b6f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:55.991531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea8c86-ddc8-4574-bcc1-2f8a968b6f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:55.993708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea8c86-ddc8-4574-bcc1-2f8a968b6f75 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:56.019180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 828/1698] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:56.023387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9bb98-147d-4382-8ff6-898a12406341 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:56.039434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9bb98-147d-4382-8ff6-898a12406341 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:56.051474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4b1f6-59b2-4053-ba86-81c8a644f236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:56.053124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4b1f6-59b2-4053-ba86-81c8a644f236 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:56.054892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4b1f6-59b2-4053-ba86-81c8a644f236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:56.055120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4b1f6-59b2-4053-ba86-81c8a644f236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:56.056203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4b1f6-59b2-4053-ba86-81c8a644f236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:56.060085 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4b1f6-59b2-4053-ba86-81c8a644f236 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:56.089889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 871/1699] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:56.093926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea8c86-ddc8-4574-bcc1-2f8a968b6f75 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:56.115487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4b1f6-59b2-4053-ba86-81c8a644f236 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:56.126056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-75b4b1f6-59b2-4053-ba86-81c8a644f236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:56.129923 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea8c86-ddc8-4574-bcc1-2f8a968b6f75 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:56.163178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bb5a9-cc6a-4639-8709-fa0c527ea1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:56.180148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bb5a9-cc6a-4639-8709-fa0c527ea1fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:56.180148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bb5a9-cc6a-4639-8709-fa0c527ea1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:56.180148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bb5a9-cc6a-4639-8709-fa0c527ea1fb admin admin] PATH_INFO: `/v3/projects/af62f5cd84804ccfb62ba857bd2629f1/users/47ae638614b24d8fb3bbc4481c1139bd/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:56.219855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4b1f6-59b2-4053-ba86-81c8a644f236 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:56.231987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bb5a9-cc6a-4639-8709-fa0c527ea1fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=af62f5cd84804ccfb62ba857bd2629f1, user_id=47ae638614b24d8fb3bbc4481c1139bd, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:56.235001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bb5a9-cc6a-4639-8709-fa0c527ea1fb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:56.261449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:56.262327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:56.262689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:56.263144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] PATH_INFO: `/v3/users/eaa530c977ab45039acb696a6a1d214e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:56.266329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa530c977ab45039acb696a6a1d214e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:56.275720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:30:56.275720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 01 05:30:56.275720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: Sep 01 05:30:56.286528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:56.393267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 829/1700] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:56 2022] PUT /identity/v3/projects/af62f5cd84804ccfb62ba857bd2629f1/users/47ae638614b24d8fb3bbc4481c1139bd/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:56.394876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475fed1-f3d1-483b-8b55-cd4d2d4aae21 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:56.397475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475fed1-f3d1-483b-8b55-cd4d2d4aae21 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:56.397932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475fed1-f3d1-483b-8b55-cd4d2d4aae21 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:56.406890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 830/1701] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 1 05:30:56 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:30:56.411389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671720ee-9fc9-4c02-b8d5-cf5e5b015e5a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:56.412604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671720ee-9fc9-4c02-b8d5-cf5e5b015e5a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:56.413046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671720ee-9fc9-4c02-b8d5-cf5e5b015e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:56.491698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-671720ee-9fc9-4c02-b8d5-cf5e5b015e5a None None] MFA Rules not processed for user `47ae638614b24d8fb3bbc4481c1139bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:56.496168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-671720ee-9fc9-4c02-b8d5-cf5e5b015e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:56.569831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa530c977ab45039acb696a6a1d214e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:56.590864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:56.650039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa530c977ab45039acb696a6a1d214e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:56.669911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 831/1702] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:56.678716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f975e8-fdbd-4a68-a154-95691ee329c7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:56.682660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f975e8-fdbd-4a68-a154-95691ee329c7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:56.682660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f975e8-fdbd-4a68-a154-95691ee329c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:56.694828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:56.703567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa530c977ab45039acb696a6a1d214e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:56.821164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa530c977ab45039acb696a6a1d214e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:56.821764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-31f975e8-fdbd-4a68-a154-95691ee329c7 None None] MFA Rules not processed for user `0e21910fca40479ca04e7f0418f7dd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:56.822097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa530c977ab45039acb696a6a1d214e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:56.823218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-31f975e8-fdbd-4a68-a154-95691ee329c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:56.839396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 872/1703] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:56 2022] DELETE /identity/v3/users/eaa530c977ab45039acb696a6a1d214e => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:56.847982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:56.852374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:56.962546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 832/1704] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:30:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:56.971664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bfab2-01e9-45c2-bd88-89917df9c7a5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:56.974889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bfab2-01e9-45c2-bd88-89917df9c7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:56.975449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bfab2-01e9-45c2-bd88-89917df9c7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:57.030552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-856bfab2-01e9-45c2-bd88-89917df9c7a5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:57.034502 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-856bfab2-01e9-45c2-bd88-89917df9c7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:57.066453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efd8f5-582e-42e7-a603-c7c47ae20359 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:57.079044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0c966-436b-44d7-8bcf-f0a47245978e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:57.079692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0c966-436b-44d7-8bcf-f0a47245978e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:57.079973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0c966-436b-44d7-8bcf-f0a47245978e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:57.081858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0c966-436b-44d7-8bcf-f0a47245978e admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:57.082759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0c966-436b-44d7-8bcf-f0a47245978e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:57.088510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0c966-436b-44d7-8bcf-f0a47245978e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:57.154079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 833/1705] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:57.155024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bfab2-01e9-45c2-bd88-89917df9c7a5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:57.161386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-856bfab2-01e9-45c2-bd88-89917df9c7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:57.215563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 873/1706] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:56 2022] POST /identity/v3/projects => generated 375 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:57.216660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0c966-436b-44d7-8bcf-f0a47245978e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:57.229718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-adc0c966-436b-44d7-8bcf-f0a47245978e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:57.292297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bfab2-01e9-45c2-bd88-89917df9c7a5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:57.344504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0c966-436b-44d7-8bcf-f0a47245978e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:57.354825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5a694-5366-4f54-9190-6acd68955fcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:57.355949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5a694-5366-4f54-9190-6acd68955fcf service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:57.356396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5a694-5366-4f54-9190-6acd68955fcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:57.357082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5a694-5366-4f54-9190-6acd68955fcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:57.398776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fbdbc-ffde-4e47-95cc-a963f229b062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:57.399692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fbdbc-ffde-4e47-95cc-a963f229b062 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:57.400132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fbdbc-ffde-4e47-95cc-a963f229b062 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:57.400566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fbdbc-ffde-4e47-95cc-a963f229b062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:57.422195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fbdbc-ffde-4e47-95cc-a963f229b062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:57.424050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fbdbc-ffde-4e47-95cc-a963f229b062 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:57.448249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 834/1707] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:57.449195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fbdbc-ffde-4e47-95cc-a963f229b062 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:57.537516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5a694-5366-4f54-9190-6acd68955fcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:57.539146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5a694-5366-4f54-9190-6acd68955fcf service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:57.571770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 874/1708] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:57.587124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fbdbc-ffde-4e47-95cc-a963f229b062 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:57.593018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca17c5-34b8-4069-a4f3-4824e0f7e2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:57.594494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca17c5-34b8-4069-a4f3-4824e0f7e2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:57.601016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca17c5-34b8-4069-a4f3-4824e0f7e2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:57.601016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca17c5-34b8-4069-a4f3-4824e0f7e2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:57.601016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca17c5-34b8-4069-a4f3-4824e0f7e2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:57.601016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca17c5-34b8-4069-a4f3-4824e0f7e2f1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:57.783985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 835/1709] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:57 2022] POST /identity/v3/users => generated 288 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:57.787985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5a694-5366-4f54-9190-6acd68955fcf service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:57.804903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5a694-5366-4f54-9190-6acd68955fcf service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:57.821872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2874707-5b6c-4f33-a999-d85a62ae73e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:57.822579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2874707-5b6c-4f33-a999-d85a62ae73e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:57.822934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2874707-5b6c-4f33-a999-d85a62ae73e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:57.823280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2874707-5b6c-4f33-a999-d85a62ae73e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:57.824045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2874707-5b6c-4f33-a999-d85a62ae73e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:57.828042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2874707-5b6c-4f33-a999-d85a62ae73e0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:57.875191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca17c5-34b8-4069-a4f3-4824e0f7e2f1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:57.877676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 875/1710] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:57.886869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fbeaa-9543-4a21-bb9e-5beda599afe3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:57.889587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fbeaa-9543-4a21-bb9e-5beda599afe3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:57.890591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fbeaa-9543-4a21-bb9e-5beda599afe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:57.897683 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca17c5-34b8-4069-a4f3-4824e0f7e2f1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:57.915990 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68089401-9355-47cc-8e6d-6788a3636b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:57.916707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68089401-9355-47cc-8e6d-6788a3636b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:57.917234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68089401-9355-47cc-8e6d-6788a3636b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:57.917603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68089401-9355-47cc-8e6d-6788a3636b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:57.918333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68089401-9355-47cc-8e6d-6788a3636b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:57.920166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68089401-9355-47cc-8e6d-6788a3636b23 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:57.945117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9d3fbeaa-9543-4a21-bb9e-5beda599afe3 None None] MFA Rules not processed for user `e9ad5076ef93462fbe614d7fc323935d`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:57.952765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 836/1711] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:57.957343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68089401-9355-47cc-8e6d-6788a3636b23 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:57.961705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9d3fbeaa-9543-4a21-bb9e-5beda599afe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:57.986540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68089401-9355-47cc-8e6d-6788a3636b23 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:58.001982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b0025-9e66-42af-a24b-97e59702f007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:58.013226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b0025-9e66-42af-a24b-97e59702f007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:58.013226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b0025-9e66-42af-a24b-97e59702f007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:58.013226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b0025-9e66-42af-a24b-97e59702f007 admin admin] PATH_INFO: `/v3/projects/bf01bf726fbc4d70b243695cb4d92bdf/users/5f33aaa4275c45db99ef83e5b5eb158c/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:58.027150 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b0025-9e66-42af-a24b-97e59702f007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf01bf726fbc4d70b243695cb4d92bdf, user_id=5f33aaa4275c45db99ef83e5b5eb158c, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:58.033616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b0025-9e66-42af-a24b-97e59702f007 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:58.081597 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 876/1712] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:58.146337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 837/1713] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:57 2022] PUT /identity/v3/projects/bf01bf726fbc4d70b243695cb4d92bdf/users/5f33aaa4275c45db99ef83e5b5eb158c/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:58.157105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e48e4e-e1a9-415a-a82c-599b49665203 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:58.158556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e48e4e-e1a9-415a-a82c-599b49665203 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:58.158856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e48e4e-e1a9-415a-a82c-599b49665203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:58.276505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:58.315464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-24e48e4e-e1a9-415a-a82c-599b49665203 None None] MFA Rules not processed for user `5f33aaa4275c45db99ef83e5b5eb158c`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:58.319609 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-24e48e4e-e1a9-415a-a82c-599b49665203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:58.489884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 877/1714] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:58.495386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b0025-9e66-42af-a24b-97e59702f007 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:58.708250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b0025-9e66-42af-a24b-97e59702f007 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:58.758506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e2f2e-7d89-4f29-a9d6-d52bef7197b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:58.760196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e2f2e-7d89-4f29-a9d6-d52bef7197b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:58.761709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e2f2e-7d89-4f29-a9d6-d52bef7197b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:58.762219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e2f2e-7d89-4f29-a9d6-d52bef7197b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:58.768724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e2f2e-7d89-4f29-a9d6-d52bef7197b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:58.774705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e2f2e-7d89-4f29-a9d6-d52bef7197b2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:58.897025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e48e4e-e1a9-415a-a82c-599b49665203 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:58.899383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-24e48e4e-e1a9-415a-a82c-599b49665203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:58.949937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 838/1715] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:58 2022] POST /identity/v3/projects => generated 379 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:58.954834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e2f2e-7d89-4f29-a9d6-d52bef7197b2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:58.958003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e48e4e-e1a9-415a-a82c-599b49665203 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:59.027386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:59.177773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:59.179347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:59.179987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:59.181682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] PATH_INFO: `/v3/users/65001f0ef2ca40989875e08ee912e259` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:59.184107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=65001f0ef2ca40989875e08ee912e259)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:59.191695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:59.240083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e2f2e-7d89-4f29-a9d6-d52bef7197b2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:59.257937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836593bb-9652-4ae6-ad13-e6c8b8ad217b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:59.260045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836593bb-9652-4ae6-ad13-e6c8b8ad217b admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:59.260270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836593bb-9652-4ae6-ad13-e6c8b8ad217b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:59.260545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836593bb-9652-4ae6-ad13-e6c8b8ad217b admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:59.261212 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836593bb-9652-4ae6-ad13-e6c8b8ad217b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:59.269031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836593bb-9652-4ae6-ad13-e6c8b8ad217b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:59.392665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 839/1716] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:58 2022] POST /identity/v3/users => generated 296 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:59.393508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836593bb-9652-4ae6-ad13-e6c8b8ad217b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:59.397236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-836593bb-9652-4ae6-ad13-e6c8b8ad217b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:59.451928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65001f0ef2ca40989875e08ee912e259'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:59.490839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65001f0ef2ca40989875e08ee912e259'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:59.509248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65001f0ef2ca40989875e08ee912e259'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:59.563353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836593bb-9652-4ae6-ad13-e6c8b8ad217b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:59.580110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ee3b3-fc8a-4df3-bf88-a2a15c8194fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:59.580745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ee3b3-fc8a-4df3-bf88-a2a15c8194fa service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:59.581099 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ee3b3-fc8a-4df3-bf88-a2a15c8194fa service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:59.581423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ee3b3-fc8a-4df3-bf88-a2a15c8194fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:59.583289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1b1ee3b3-fc8a-4df3-bf88-a2a15c8194fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:59.596188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ee3b3-fc8a-4df3-bf88-a2a15c8194fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:59.599385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ee3b3-fc8a-4df3-bf88-a2a15c8194fa service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:59.612929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65001f0ef2ca40989875e08ee912e259'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:59.613881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65001f0ef2ca40989875e08ee912e259'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:59.627965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 878/1717] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:58 2022] DELETE /identity/v3/users/65001f0ef2ca40989875e08ee912e259 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:59.627965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:59.628888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 840/1718] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:59.630799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10937f2d-4d99-46c6-8268-de50b941593e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:59.631812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:59.632538 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10937f2d-4d99-46c6-8268-de50b941593e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:59.632823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10937f2d-4d99-46c6-8268-de50b941593e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:59.708006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-10937f2d-4d99-46c6-8268-de50b941593e None None] MFA Rules not processed for user `fb50b15e44814c6b82b2a75e3c74de0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:59.715207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-10937f2d-4d99-46c6-8268-de50b941593e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:59.792607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482678a1-c9bd-498a-ac89-206b530d2c3d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:59.803186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b82169-7a9c-4ee8-9b23-d965572f6ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:59.803864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b82169-7a9c-4ee8-9b23-d965572f6ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:59.804143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b82169-7a9c-4ee8-9b23-d965572f6ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:59.804412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b82169-7a9c-4ee8-9b23-d965572f6ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:59.805061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b82169-7a9c-4ee8-9b23-d965572f6ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:59.807485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b82169-7a9c-4ee8-9b23-d965572f6ef6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:59.838890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 841/1719] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:59.848343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 879/1720] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:59.849113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c79900-b888-4433-bd2a-19e35760b4b1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:59.849206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8c29e-0255-4c48-b98b-aba36240b638 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:59.849206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8c29e-0255-4c48-b98b-aba36240b638 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:59.849345 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c79900-b888-4433-bd2a-19e35760b4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:59.849345 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c79900-b888-4433-bd2a-19e35760b4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:59.849517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8c29e-0255-4c48-b98b-aba36240b638 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:59.857536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 880/1721] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 1 05:30:59 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:30:59.860594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8c29e-0255-4c48-b98b-aba36240b638 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:59.906538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8c29e-0255-4c48-b98b-aba36240b638 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:59.918770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-00c79900-b888-4433-bd2a-19e35760b4b1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:59.919569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29ca3-98fd-43a3-99c0-23010d27c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:59.920475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29ca3-98fd-43a3-99c0-23010d27c20c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:59.921641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29ca3-98fd-43a3-99c0-23010d27c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:59.924910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29ca3-98fd-43a3-99c0-23010d27c20c admin admin] PATH_INFO: `/v3/projects/1195c2b2b55546bca8a0e441a73d5210/users/b8fb449e8d3349c9a0d5bf5f55d2c992/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:59.925038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-00c79900-b888-4433-bd2a-19e35760b4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:59.962888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf29ca3-98fd-43a3-99c0-23010d27c20c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1195c2b2b55546bca8a0e441a73d5210, user_id=b8fb449e8d3349c9a0d5bf5f55d2c992, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:59.965674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf29ca3-98fd-43a3-99c0-23010d27c20c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:59.966964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 842/1722] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:59.970362 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7995fc-814f-48be-8345-11eb56656c58 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:59.970905 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7995fc-814f-48be-8345-11eb56656c58 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:59.973584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7995fc-814f-48be-8345-11eb56656c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:00.012337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5a7995fc-814f-48be-8345-11eb56656c58 None None] MFA Rules not processed for user `fb50b15e44814c6b82b2a75e3c74de0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:00.012337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5a7995fc-814f-48be-8345-11eb56656c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:00.033824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 843/1723] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:30:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:00.041014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099a94d-1ed4-454a-b0ab-9dc3ee21666e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:00.041014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099a94d-1ed4-454a-b0ab-9dc3ee21666e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:00.041014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099a94d-1ed4-454a-b0ab-9dc3ee21666e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:00.061001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8099a94d-1ed4-454a-b0ab-9dc3ee21666e None None] MFA Rules not processed for user `0e21910fca40479ca04e7f0418f7dd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:00.062472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8099a94d-1ed4-454a-b0ab-9dc3ee21666e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:00.066182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 881/1724] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:59 2022] PUT /identity/v3/projects/1195c2b2b55546bca8a0e441a73d5210/users/b8fb449e8d3349c9a0d5bf5f55d2c992/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:00.067221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29ca3-98fd-43a3-99c0-23010d27c20c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:00.072291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dcf29ca3-98fd-43a3-99c0-23010d27c20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:00.136317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 844/1725] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:31:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:00.137400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099a94d-1ed4-454a-b0ab-9dc3ee21666e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:00.138853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8099a94d-1ed4-454a-b0ab-9dc3ee21666e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:00.203750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29ca3-98fd-43a3-99c0-23010d27c20c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:00.213585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd5c56-48bb-43db-9194-75d5b63759e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:00.214220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd5c56-48bb-43db-9194-75d5b63759e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:00.214499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd5c56-48bb-43db-9194-75d5b63759e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:00.214774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd5c56-48bb-43db-9194-75d5b63759e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:00.329300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099a94d-1ed4-454a-b0ab-9dc3ee21666e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:00.348495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa377d13-15c5-4a06-9ec9-3089593e3c93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:00.349242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa377d13-15c5-4a06-9ec9-3089593e3c93 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:00.353145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa377d13-15c5-4a06-9ec9-3089593e3c93 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:00.353145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa377d13-15c5-4a06-9ec9-3089593e3c93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:00.384502 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd5c56-48bb-43db-9194-75d5b63759e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:00.387579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd5c56-48bb-43db-9194-75d5b63759e5 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:00.465701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:00.475752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 882/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:00.475752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd5c56-48bb-43db-9194-75d5b63759e5 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:00.475752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1cbd5c56-48bb-43db-9194-75d5b63759e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:00.503574 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd5c56-48bb-43db-9194-75d5b63759e5 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:00.522170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:00.522758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:00.523301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:00.523928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa377d13-15c5-4a06-9ec9-3089593e3c93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:00.524417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] PATH_INFO: `/v3/projects/796add2986054d78ad461b63a33d9e09` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:00.526033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=796add2986054d78ad461b63a33d9e09)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:00.527945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa377d13-15c5-4a06-9ec9-3089593e3c93 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:00.534205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:00.548360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 845/1727] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:00.548970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa377d13-15c5-4a06-9ec9-3089593e3c93 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:00.576018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa377d13-15c5-4a06-9ec9-3089593e3c93 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:00.609343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af64c1-61c8-4c72-8036-f7b390741477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:00.615886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af64c1-61c8-4c72-8036-f7b390741477 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:00.616181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af64c1-61c8-4c72-8036-f7b390741477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:00.618042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af64c1-61c8-4c72-8036-f7b390741477 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:00.624038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af64c1-61c8-4c72-8036-f7b390741477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:00.628816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af64c1-61c8-4c72-8036-f7b390741477 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:00.658796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 846/1728] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:00.662260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af64c1-61c8-4c72-8036-f7b390741477 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:00.669112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-00af64c1-61c8-4c72-8036-f7b390741477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:00.688237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af64c1-61c8-4c72-8036-f7b390741477 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:00.696297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85993c2b-0633-423b-98ac-d08c2e0a9c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:00.697294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85993c2b-0633-423b-98ac-d08c2e0a9c23 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:00.698087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85993c2b-0633-423b-98ac-d08c2e0a9c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:00.698637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85993c2b-0633-423b-98ac-d08c2e0a9c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:00.710638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85993c2b-0633-423b-98ac-d08c2e0a9c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:00.718938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85993c2b-0633-423b-98ac-d08c2e0a9c23 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:00.747644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 847/1729] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:00.749070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709ac23-781e-4c66-8508-1f5f17889e0c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:00.749360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709ac23-781e-4c66-8508-1f5f17889e0c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:00.752455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709ac23-781e-4c66-8508-1f5f17889e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:00.786740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4709ac23-781e-4c66-8508-1f5f17889e0c None None] MFA Rules not processed for user `b9db0f3de10842eb97dc6fc580f4489a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:00.788492 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4709ac23-781e-4c66-8508-1f5f17889e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:00.789770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796add2986054d78ad461b63a33d9e09', 'request_id': 'req-a5aba1c3-0c9a-48fc-9347-144aec829c52'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:00.860762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:00.924693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796add2986054d78ad461b63a33d9e09', 'request_id': 'req-a5aba1c3-0c9a-48fc-9347-144aec829c52'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:00.937356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796add2986054d78ad461b63a33d9e09', 'request_id': 'req-a5aba1c3-0c9a-48fc-9347-144aec829c52'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:00.942057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 848/1730] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:00.942057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709ac23-781e-4c66-8508-1f5f17889e0c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:00.986951 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] The token cache is being invalidate because project 796add2986054d78ad461b63a33d9e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:00.987253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:00.990000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:00.994908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 883/1731] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:00 2022] DELETE /identity/v3/projects/796add2986054d78ad461b63a33d9e09 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:00.999472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:01.019029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:01.142837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709ac23-781e-4c66-8508-1f5f17889e0c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:01.157967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfcfed-13b0-4d6f-a259-24e6a23c20d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:01.160261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfcfed-13b0-4d6f-a259-24e6a23c20d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:01.162032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfcfed-13b0-4d6f-a259-24e6a23c20d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:01.162032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfcfed-13b0-4d6f-a259-24e6a23c20d3 admin admin] PATH_INFO: `/v3/projects/1195c2b2b55546bca8a0e441a73d5210/users/b8fb449e8d3349c9a0d5bf5f55d2c992/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:01.164072 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aba1c3-0c9a-48fc-9347-144aec829c52 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:01.167779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfcfed-13b0-4d6f-a259-24e6a23c20d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1195c2b2b55546bca8a0e441a73d5210, user_id=b8fb449e8d3349c9a0d5bf5f55d2c992, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:01.176777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:01.177233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfcfed-13b0-4d6f-a259-24e6a23c20d3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:01.177734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:01.178130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:01.178470 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] PATH_INFO: `/v3/users/738e5799aa16425d81969bcc914e2a55` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:01.181305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=738e5799aa16425d81969bcc914e2a55)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:01.183438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:01.209527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 849/1732] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:00 2022] PUT /identity/v3/projects/1195c2b2b55546bca8a0e441a73d5210/users/b8fb449e8d3349c9a0d5bf5f55d2c992/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:01.217135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea905a9-f87f-4e0e-8533-e704ada097ab None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:01.218527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea905a9-f87f-4e0e-8533-e704ada097ab None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:01.218527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea905a9-f87f-4e0e-8533-e704ada097ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:01.274490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8ea905a9-f87f-4e0e-8533-e704ada097ab None None] MFA Rules not processed for user `b8fb449e8d3349c9a0d5bf5f55d2c992`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:01.275876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8ea905a9-f87f-4e0e-8533-e704ada097ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:01.353177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:01.439586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 850/1733] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:01.439586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea905a9-f87f-4e0e-8533-e704ada097ab None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:01.439586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8ea905a9-f87f-4e0e-8533-e704ada097ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:01.472320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '738e5799aa16425d81969bcc914e2a55'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:01.505781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '738e5799aa16425d81969bcc914e2a55'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:01.524866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '738e5799aa16425d81969bcc914e2a55'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:01.598300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea905a9-f87f-4e0e-8533-e704ada097ab None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:01.615849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:01.616363 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:01.616658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:01.616925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] PATH_INFO: `/v3/projects/e443b5bb7bd64e6da3e868f8e7a4208d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:01.617322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '738e5799aa16425d81969bcc914e2a55'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:01.618573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e443b5bb7bd64e6da3e868f8e7a4208d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:01.618820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '738e5799aa16425d81969bcc914e2a55'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:01.620982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:01.637379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 884/1734] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:00 2022] DELETE /identity/v3/users/738e5799aa16425d81969bcc914e2a55 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:01.642291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:01.644772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:01.703506 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:01.806394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e443b5bb7bd64e6da3e868f8e7a4208d', 'request_id': 'req-b30fc317-a652-4c1d-933d-2b8615232589'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:01.843318 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bb302-d801-47ea-910e-05a5e26f6f52 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:01.913428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e443b5bb7bd64e6da3e868f8e7a4208d', 'request_id': 'req-b30fc317-a652-4c1d-933d-2b8615232589'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:01.929553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e443b5bb7bd64e6da3e868f8e7a4208d', 'request_id': 'req-b30fc317-a652-4c1d-933d-2b8615232589'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:01.954800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3a2ed-a27d-4c64-9ec7-eb29db839fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:01.955541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a2ed-a27d-4c64-9ec7-eb29db839fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:01.955940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a2ed-a27d-4c64-9ec7-eb29db839fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:01.956424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a2ed-a27d-4c64-9ec7-eb29db839fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:01.958758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-46f3a2ed-a27d-4c64-9ec7-eb29db839fd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:01.959337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] The token cache is being invalidate because project e443b5bb7bd64e6da3e868f8e7a4208d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:01.959633 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:01.974455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b30fc317-a652-4c1d-933d-2b8615232589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:01.978937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 851/1735] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:01 2022] DELETE /identity/v3/projects/e443b5bb7bd64e6da3e868f8e7a4208d => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:01.987695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa71d-53d0-4db6-bba1-cb1ef232c7b7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:01.987995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa71d-53d0-4db6-bba1-cb1ef232c7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:01.988314 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa71d-53d0-4db6-bba1-cb1ef232c7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:02.103795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3a2ed-a27d-4c64-9ec7-eb29db839fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:02.105687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3a2ed-a27d-4c64-9ec7-eb29db839fd6 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:02.114726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0b4fa71d-53d0-4db6-bba1-cb1ef232c7b7 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:02.114726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0b4fa71d-53d0-4db6-bba1-cb1ef232c7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:02.119324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 885/1736] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:02.292959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3a2ed-a27d-4c64-9ec7-eb29db839fd6 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:02.298602 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-46f3a2ed-a27d-4c64-9ec7-eb29db839fd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:02.327562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3a2ed-a27d-4c64-9ec7-eb29db839fd6 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:02.350028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0210a80-c819-4fb7-8087-997d0b71fed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:02.350028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0210a80-c819-4fb7-8087-997d0b71fed3 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:02.350028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0210a80-c819-4fb7-8087-997d0b71fed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:02.350028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0210a80-c819-4fb7-8087-997d0b71fed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:02.363144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c0210a80-c819-4fb7-8087-997d0b71fed3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:02.439559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 852/1737] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:02.455711 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658093ce-5f80-471e-a93a-44f3fc6d63b6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:02.455995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658093ce-5f80-471e-a93a-44f3fc6d63b6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:02.456271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658093ce-5f80-471e-a93a-44f3fc6d63b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:02.587817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-658093ce-5f80-471e-a93a-44f3fc6d63b6 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:02.594106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-658093ce-5f80-471e-a93a-44f3fc6d63b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:02.614518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 853/1738] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:02.618744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658093ce-5f80-471e-a93a-44f3fc6d63b6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:02.659640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658093ce-5f80-471e-a93a-44f3fc6d63b6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:02.661739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0210a80-c819-4fb7-8087-997d0b71fed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:02.670226 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0210a80-c819-4fb7-8087-997d0b71fed3 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:02.687369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7212caa-6f00-47ec-ab2f-a464f4672093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:02.688051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7212caa-6f00-47ec-ab2f-a464f4672093 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:02.688387 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7212caa-6f00-47ec-ab2f-a464f4672093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:02.692829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7212caa-6f00-47ec-ab2f-a464f4672093 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:02.693536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7212caa-6f00-47ec-ab2f-a464f4672093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:02.699542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7212caa-6f00-47ec-ab2f-a464f4672093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:02.699542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7212caa-6f00-47ec-ab2f-a464f4672093 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:02.702403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 886/1739] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:02.720623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0210a80-c819-4fb7-8087-997d0b71fed3 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:02.723543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c0210a80-c819-4fb7-8087-997d0b71fed3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:02.737343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 854/1740] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:31:02.741482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7212caa-6f00-47ec-ab2f-a464f4672093 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:02.759233 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0210a80-c819-4fb7-8087-997d0b71fed3 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:02.762450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7212caa-6f00-47ec-ab2f-a464f4672093 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:02.776150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:02.777124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:02.777303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:02.777619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] PATH_INFO: `/v3/projects/60e47bd23f144969afd1d335826c9fda` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:02.779479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=60e47bd23f144969afd1d335826c9fda)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:02.779630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d04256-6b03-47a8-896d-40594039a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:02.780690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d04256-6b03-47a8-896d-40594039a769 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:02.781353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d04256-6b03-47a8-896d-40594039a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:02.781951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d04256-6b03-47a8-896d-40594039a769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:02.782214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:02.782971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d04256-6b03-47a8-896d-40594039a769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:02.788540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d04256-6b03-47a8-896d-40594039a769 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:02.994277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 855/1741] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:02 2022] POST /identity/v3/projects => generated 403 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:02.995474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:02.999204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d04256-6b03-47a8-896d-40594039a769 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:03.038377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60e47bd23f144969afd1d335826c9fda', 'request_id': 'req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:03.117400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d04256-6b03-47a8-896d-40594039a769 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:03.161008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60e47bd23f144969afd1d335826c9fda', 'request_id': 'req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:03.176757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60e47bd23f144969afd1d335826c9fda', 'request_id': 'req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:03.208009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] The token cache is being invalidate because project 60e47bd23f144969afd1d335826c9fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:03.208305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:03.210918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a483f76-06e3-48f5-8aa5-1bf2193e55cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:03.221804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 887/1742] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:02 2022] DELETE /identity/v3/projects/60e47bd23f144969afd1d335826c9fda => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:03.223288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572f043-2da0-4d5f-8031-228c34792801 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:03.223768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572f043-2da0-4d5f-8031-228c34792801 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:03.224046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572f043-2da0-4d5f-8031-228c34792801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:03.276921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9572f043-2da0-4d5f-8031-228c34792801 None None] MFA Rules not processed for user `fb50b15e44814c6b82b2a75e3c74de0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:03.280596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9572f043-2da0-4d5f-8031-228c34792801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:03.304364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe0b69-dfba-4203-8ff2-1aeff3edc74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:03.305544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe0b69-dfba-4203-8ff2-1aeff3edc74e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:03.305544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe0b69-dfba-4203-8ff2-1aeff3edc74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:03.305831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe0b69-dfba-4203-8ff2-1aeff3edc74e admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:03.306615 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe0b69-dfba-4203-8ff2-1aeff3edc74e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:03.309137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe0b69-dfba-4203-8ff2-1aeff3edc74e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:03.379676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 888/1743] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:03.382409 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030720-6589-458f-afee-3878c2039373 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:03.382703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030720-6589-458f-afee-3878c2039373 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:03.384642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030720-6589-458f-afee-3878c2039373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:03.440297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e7030720-6589-458f-afee-3878c2039373 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:03.442034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e7030720-6589-458f-afee-3878c2039373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:03.455065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 856/1744] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:02 2022] POST /identity/v3/users => generated 302 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:03.456887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe0b69-dfba-4203-8ff2-1aeff3edc74e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:03.464026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5fbe0b69-dfba-4203-8ff2-1aeff3edc74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:03.464535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 889/1745] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:03.466418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dbbb6-2654-4b23-8eff-ea8609657c0a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:03.466882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dbbb6-2654-4b23-8eff-ea8609657c0a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:03.467298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dbbb6-2654-4b23-8eff-ea8609657c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:03.492652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe0b69-dfba-4203-8ff2-1aeff3edc74e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:03.501288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef0e62-246c-4926-ac08-c1d468eaab5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:03.507927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef0e62-246c-4926-ac08-c1d468eaab5f admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:03.507927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef0e62-246c-4926-ac08-c1d468eaab5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:03.507927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef0e62-246c-4926-ac08-c1d468eaab5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:03.507927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef0e62-246c-4926-ac08-c1d468eaab5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:03.507927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef0e62-246c-4926-ac08-c1d468eaab5f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:03.515068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5e1dbbb6-2654-4b23-8eff-ea8609657c0a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:03.516868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5e1dbbb6-2654-4b23-8eff-ea8609657c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:03.523042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 857/1746] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:03.529884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 890/1747] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:03.536658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef0e62-246c-4926-ac08-c1d468eaab5f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:03.541076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dbbb6-2654-4b23-8eff-ea8609657c0a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:03.547655 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef0e62-246c-4926-ac08-c1d468eaab5f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:03.552915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dbbb6-2654-4b23-8eff-ea8609657c0a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:03.556370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae8d87-3446-424f-9b74-448b96aac696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:03.557498 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae8d87-3446-424f-9b74-448b96aac696 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:03.558068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae8d87-3446-424f-9b74-448b96aac696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:03.558599 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae8d87-3446-424f-9b74-448b96aac696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:03.559870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca274e-3553-445b-9e2b-5baf8f9606a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:03.560774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca274e-3553-445b-9e2b-5baf8f9606a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:03.561541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca274e-3553-445b-9e2b-5baf8f9606a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:03.561826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca274e-3553-445b-9e2b-5baf8f9606a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:03.562499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca274e-3553-445b-9e2b-5baf8f9606a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:03.562804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca274e-3553-445b-9e2b-5baf8f9606a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:03.565565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae8d87-3446-424f-9b74-448b96aac696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:03.565740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca274e-3553-445b-9e2b-5baf8f9606a9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:03.567870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae8d87-3446-424f-9b74-448b96aac696 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:03.590998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 891/1748] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:31:03.592727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 858/1749] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:03.597775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca274e-3553-445b-9e2b-5baf8f9606a9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:03.600880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae8d87-3446-424f-9b74-448b96aac696 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:03.627479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca274e-3553-445b-9e2b-5baf8f9606a9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:03.639075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6d46e-ce38-42b0-8c3e-b505d1c138f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:03.641811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:03.643018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6d46e-ce38-42b0-8c3e-b505d1c138f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:03.643452 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6d46e-ce38-42b0-8c3e-b505d1c138f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:03.643897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6d46e-ce38-42b0-8c3e-b505d1c138f6 admin admin] PATH_INFO: `/v3/projects/b2c18a7afbc0429d8f5c1c4b8ab4137a/users/d1b641a6bdd54994b536958d04962aa3/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:03.648943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae8d87-3446-424f-9b74-448b96aac696 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:03.665080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4c245-d9be-474b-8dc6-a25abc34761f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:03.666494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4c245-d9be-474b-8dc6-a25abc34761f admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:03.666771 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4c245-d9be-474b-8dc6-a25abc34761f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:03.667041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4c245-d9be-474b-8dc6-a25abc34761f admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:03.667687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4c245-d9be-474b-8dc6-a25abc34761f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:03.672135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4c245-d9be-474b-8dc6-a25abc34761f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:03.677807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6d46e-ce38-42b0-8c3e-b505d1c138f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c18a7afbc0429d8f5c1c4b8ab4137a, user_id=d1b641a6bdd54994b536958d04962aa3, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:03.681998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6d46e-ce38-42b0-8c3e-b505d1c138f6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:03.819506 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 892/1750] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:03 2022] PUT /identity/v3/projects/b2c18a7afbc0429d8f5c1c4b8ab4137a/users/d1b641a6bdd54994b536958d04962aa3/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:03.832697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 859/1751] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:03 2022] POST /identity/v3/projects => generated 381 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:03.838569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4c245-d9be-474b-8dc6-a25abc34761f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:03.839686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbc8fb-a75d-444f-8b51-3141ee60189a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:03.840182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbc8fb-a75d-444f-8b51-3141ee60189a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:03.843655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbc8fb-a75d-444f-8b51-3141ee60189a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:03.922682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6bfbc8fb-a75d-444f-8b51-3141ee60189a None None] MFA Rules not processed for user `d1b641a6bdd54994b536958d04962aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:03.924333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6bfbc8fb-a75d-444f-8b51-3141ee60189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:03.999641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4c245-d9be-474b-8dc6-a25abc34761f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:04.002961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf0eca-08e4-486c-aec2-85249fda385f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:04.003856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf0eca-08e4-486c-aec2-85249fda385f admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:04.004144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf0eca-08e4-486c-aec2-85249fda385f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:04.004424 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf0eca-08e4-486c-aec2-85249fda385f admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:04.005121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf0eca-08e4-486c-aec2-85249fda385f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:04.007398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf0eca-08e4-486c-aec2-85249fda385f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:04.026496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 893/1752] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:03 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:04.031171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbc8fb-a75d-444f-8b51-3141ee60189a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:04.046641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbc8fb-a75d-444f-8b51-3141ee60189a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:04.067402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662b943-4b31-40d5-a441-0048af21aeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:04.068294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662b943-4b31-40d5-a441-0048af21aeec admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:04.068781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662b943-4b31-40d5-a441-0048af21aeec admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:04.069238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662b943-4b31-40d5-a441-0048af21aeec admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:04.070308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662b943-4b31-40d5-a441-0048af21aeec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:04.072428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662b943-4b31-40d5-a441-0048af21aeec admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:04.144458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:04.151417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 860/1753] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:03 2022] POST /identity/v3/users => generated 291 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:04.157295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf0eca-08e4-486c-aec2-85249fda385f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:04.194315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf0eca-08e4-486c-aec2-85249fda385f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:04.206243 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8a7f-e8cc-4df4-8209-eb34908857a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:04.211525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8a7f-e8cc-4df4-8209-eb34908857a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:04.211525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8a7f-e8cc-4df4-8209-eb34908857a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:04.211525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8a7f-e8cc-4df4-8209-eb34908857a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:04.211525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8a7f-e8cc-4df4-8209-eb34908857a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:04.212903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 894/1754] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:04 2022] POST /identity/v3/projects => generated 405 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:04.216934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662b943-4b31-40d5-a441-0048af21aeec admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:04.219911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8a7f-e8cc-4df4-8209-eb34908857a3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:04.234193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 861/1755] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:04.238565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8a7f-e8cc-4df4-8209-eb34908857a3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:04.251537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8a7f-e8cc-4df4-8209-eb34908857a3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:04.372438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9483d0-c27b-43f3-91a3-7b787441f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:04.377858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9483d0-c27b-43f3-91a3-7b787441f1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:04.377858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9483d0-c27b-43f3-91a3-7b787441f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:04.377858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9483d0-c27b-43f3-91a3-7b787441f1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:04.377858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9483d0-c27b-43f3-91a3-7b787441f1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:04.377858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9483d0-c27b-43f3-91a3-7b787441f1f4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:04.379994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662b943-4b31-40d5-a441-0048af21aeec admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:04.395715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a7ce1-ee15-4d82-be95-534b7a32e4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:04.398131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a7ce1-ee15-4d82-be95-534b7a32e4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:04.398983 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a7ce1-ee15-4d82-be95-534b7a32e4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:04.399856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a7ce1-ee15-4d82-be95-534b7a32e4c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:04.401207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a7ce1-ee15-4d82-be95-534b7a32e4c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:04.423645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 862/1756] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:04.425019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a7ce1-ee15-4d82-be95-534b7a32e4c5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:04.425484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9483d0-c27b-43f3-91a3-7b787441f1f4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:04.479881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9483d0-c27b-43f3-91a3-7b787441f1f4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:04.502826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5c54a-b409-4436-be19-25f2d0a6fccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:04.502826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5c54a-b409-4436-be19-25f2d0a6fccd admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:04.502826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5c54a-b409-4436-be19-25f2d0a6fccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:04.502826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5c54a-b409-4436-be19-25f2d0a6fccd admin admin] PATH_INFO: `/v3/projects/7393ba3c406b4e0096584ce1b7234502/users/c038fb28b3b445669823d1170c6eabf1/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:04.526904 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:04.536174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5c54a-b409-4436-be19-25f2d0a6fccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7393ba3c406b4e0096584ce1b7234502, user_id=c038fb28b3b445669823d1170c6eabf1, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:04.542436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5c54a-b409-4436-be19-25f2d0a6fccd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:04.729430 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 895/1757] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:04 2022] POST /identity/v3/users => generated 309 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:04.731025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a7ce1-ee15-4d82-be95-534b7a32e4c5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:04.758692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 863/1758] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:04 2022] PUT /identity/v3/projects/7393ba3c406b4e0096584ce1b7234502/users/c038fb28b3b445669823d1170c6eabf1/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:04.760387 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64f20d-705f-4b8d-bfe7-c611806b06d5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:04.766958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64f20d-705f-4b8d-bfe7-c611806b06d5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:04.767953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a7ce1-ee15-4d82-be95-534b7a32e4c5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:04.768647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64f20d-705f-4b8d-bfe7-c611806b06d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:04.893161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1179e9-27f9-4e76-a0e4-e7dea6cb299e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:04.893435 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1179e9-27f9-4e76-a0e4-e7dea6cb299e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:04.893903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1179e9-27f9-4e76-a0e4-e7dea6cb299e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:04.894423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1179e9-27f9-4e76-a0e4-e7dea6cb299e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:04.895216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1179e9-27f9-4e76-a0e4-e7dea6cb299e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:04.896950 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1179e9-27f9-4e76-a0e4-e7dea6cb299e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:04.909706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 896/1759] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:04.920989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1179e9-27f9-4e76-a0e4-e7dea6cb299e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:04.921596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9f64f20d-705f-4b8d-bfe7-c611806b06d5 None None] MFA Rules not processed for user `c038fb28b3b445669823d1170c6eabf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:04.923058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9f64f20d-705f-4b8d-bfe7-c611806b06d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:04.963232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1179e9-27f9-4e76-a0e4-e7dea6cb299e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:05.054402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0da2e-e7da-4a2c-89a0-cbdc0649dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:05.068166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0da2e-e7da-4a2c-89a0-cbdc0649dcb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:05.071100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0da2e-e7da-4a2c-89a0-cbdc0649dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:05.071620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0da2e-e7da-4a2c-89a0-cbdc0649dcb4 admin admin] PATH_INFO: `/v3/projects/9608c76db6a64fc9b9441e4f5b9f162d/users/bea4f8dc19ff4502a38d709e468c97b0/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:05.081754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 864/1760] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:05.089901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64f20d-705f-4b8d-bfe7-c611806b06d5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:05.105791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64f20d-705f-4b8d-bfe7-c611806b06d5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:05.112681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c66be5-1af3-4221-89d8-6f189319f750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:05.113641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c66be5-1af3-4221-89d8-6f189319f750 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:05.114153 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c66be5-1af3-4221-89d8-6f189319f750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:05.114741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c66be5-1af3-4221-89d8-6f189319f750 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:05.115808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c66be5-1af3-4221-89d8-6f189319f750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:05.118109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c66be5-1af3-4221-89d8-6f189319f750 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:05.182836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0da2e-e7da-4a2c-89a0-cbdc0649dcb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9608c76db6a64fc9b9441e4f5b9f162d, user_id=bea4f8dc19ff4502a38d709e468c97b0, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:05.199364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0da2e-e7da-4a2c-89a0-cbdc0649dcb4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:05.253765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 865/1761] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:05 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:05.258017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c66be5-1af3-4221-89d8-6f189319f750 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:05.343173 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 897/1762] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:04 2022] PUT /identity/v3/projects/9608c76db6a64fc9b9441e4f5b9f162d/users/bea4f8dc19ff4502a38d709e468c97b0/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:05.347274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0da2e-e7da-4a2c-89a0-cbdc0649dcb4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:05.492426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c66be5-1af3-4221-89d8-6f189319f750 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:05.501444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0da2e-e7da-4a2c-89a0-cbdc0649dcb4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:05.507469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d1f8c-69f6-4279-ab7c-4ae906f54b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:05.508108 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d1f8c-69f6-4279-ab7c-4ae906f54b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:05.508433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d1f8c-69f6-4279-ab7c-4ae906f54b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:05.508862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d1f8c-69f6-4279-ab7c-4ae906f54b39 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:05.509584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d1f8c-69f6-4279-ab7c-4ae906f54b39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:05.513792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d1f8c-69f6-4279-ab7c-4ae906f54b39 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:05.518793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86c5c7-a35b-469b-accf-240e26ccdbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:05.518793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86c5c7-a35b-469b-accf-240e26ccdbb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:05.518793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86c5c7-a35b-469b-accf-240e26ccdbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:05.518793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86c5c7-a35b-469b-accf-240e26ccdbb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:05.519257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86c5c7-a35b-469b-accf-240e26ccdbb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:05.523500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86c5c7-a35b-469b-accf-240e26ccdbb0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:05.552134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 898/1763] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:05.556623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86c5c7-a35b-469b-accf-240e26ccdbb0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:05.578504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86c5c7-a35b-469b-accf-240e26ccdbb0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:05.587205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676bf06-d476-4851-873b-0044e5d688f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:05.588066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676bf06-d476-4851-873b-0044e5d688f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:05.588396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676bf06-d476-4851-873b-0044e5d688f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:05.589091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676bf06-d476-4851-873b-0044e5d688f7 admin admin] PATH_INFO: `/v3/projects/9608c76db6a64fc9b9441e4f5b9f162d/users/bea4f8dc19ff4502a38d709e468c97b0/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:05.591337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676bf06-d476-4851-873b-0044e5d688f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9608c76db6a64fc9b9441e4f5b9f162d, user_id=bea4f8dc19ff4502a38d709e468c97b0, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:05.597031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676bf06-d476-4851-873b-0044e5d688f7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:05.611134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 899/1764] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:05 2022] PUT /identity/v3/projects/9608c76db6a64fc9b9441e4f5b9f162d/users/bea4f8dc19ff4502a38d709e468c97b0/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:05.630509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c565e6-27a7-43ca-8f49-5e78339d6a45 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:05.630870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c565e6-27a7-43ca-8f49-5e78339d6a45 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:05.631210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c565e6-27a7-43ca-8f49-5e78339d6a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:05.662113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e6c565e6-27a7-43ca-8f49-5e78339d6a45 None None] MFA Rules not processed for user `bea4f8dc19ff4502a38d709e468c97b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:05.663334 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e6c565e6-27a7-43ca-8f49-5e78339d6a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:05.696601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 866/1765] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:05 2022] POST /identity/v3/users => generated 297 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:05.696601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d1f8c-69f6-4279-ab7c-4ae906f54b39 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:05.709772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d1f8c-69f6-4279-ab7c-4ae906f54b39 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:05.886159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbecaa9-f246-48d7-a74b-018d9280b542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:05.886159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbecaa9-f246-48d7-a74b-018d9280b542 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:05.886159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbecaa9-f246-48d7-a74b-018d9280b542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:05.886159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbecaa9-f246-48d7-a74b-018d9280b542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:05.891582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbecaa9-f246-48d7-a74b-018d9280b542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:05.895601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbecaa9-f246-48d7-a74b-018d9280b542 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:05.927567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 867/1766] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:05.930523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbecaa9-f246-48d7-a74b-018d9280b542 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:05.945512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 900/1767] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:05 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:05.950020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbecaa9-f246-48d7-a74b-018d9280b542 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:05.958308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c565e6-27a7-43ca-8f49-5e78339d6a45 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:05.961312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb14dfc-3059-4e5a-9d61-ada8e57cc135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:05.964012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb14dfc-3059-4e5a-9d61-ada8e57cc135 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:05.964685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb14dfc-3059-4e5a-9d61-ada8e57cc135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:05.965210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb14dfc-3059-4e5a-9d61-ada8e57cc135 admin admin] PATH_INFO: `/v3/projects/eae05b5b21524eb58d8e6f47062610d0/users/30096dbf807242d2bd1c82b37c5d98b5/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:05.970334 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e6c565e6-27a7-43ca-8f49-5e78339d6a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:06.011590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb14dfc-3059-4e5a-9d61-ada8e57cc135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae05b5b21524eb58d8e6f47062610d0, user_id=30096dbf807242d2bd1c82b37c5d98b5, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:06.016785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb14dfc-3059-4e5a-9d61-ada8e57cc135 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:06.151086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 868/1768] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:05 2022] PUT /identity/v3/projects/eae05b5b21524eb58d8e6f47062610d0/users/30096dbf807242d2bd1c82b37c5d98b5/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:06.155460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb14dfc-3059-4e5a-9d61-ada8e57cc135 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:06.188377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c565e6-27a7-43ca-8f49-5e78339d6a45 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:06.420048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2852b8-24f7-4243-819e-aab3cef2fbaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:06.421991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2852b8-24f7-4243-819e-aab3cef2fbaf service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:06.421991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2852b8-24f7-4243-819e-aab3cef2fbaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:06.421991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2852b8-24f7-4243-819e-aab3cef2fbaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:06.487649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb14dfc-3059-4e5a-9d61-ada8e57cc135 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:06.509016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b9d4e-48a4-425c-9308-ebab1d1ea8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:06.511336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9d4e-48a4-425c-9308-ebab1d1ea8de admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:06.511616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9d4e-48a4-425c-9308-ebab1d1ea8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:06.511883 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9d4e-48a4-425c-9308-ebab1d1ea8de admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:06.512534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b9d4e-48a4-425c-9308-ebab1d1ea8de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:06.514308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b9d4e-48a4-425c-9308-ebab1d1ea8de admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:06.556278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 869/1769] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:06.561155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5668c-c750-45ab-907b-cc24d0dea968 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:06.562264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5668c-c750-45ab-907b-cc24d0dea968 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:06.568442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5668c-c750-45ab-907b-cc24d0dea968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:06.655306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2852b8-24f7-4243-819e-aab3cef2fbaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:06.655306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2852b8-24f7-4243-819e-aab3cef2fbaf service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:06.668801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 901/1770] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:05 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:06.671327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2852b8-24f7-4243-819e-aab3cef2fbaf service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:06.722586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0fa5668c-c750-45ab-907b-cc24d0dea968 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:06.735649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0fa5668c-c750-45ab-907b-cc24d0dea968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:06.758662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2852b8-24f7-4243-819e-aab3cef2fbaf service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:06.783160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b991960-21ce-4ff1-9641-83e1e20eebcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:06.784657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b991960-21ce-4ff1-9641-83e1e20eebcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:06.785651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b991960-21ce-4ff1-9641-83e1e20eebcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:06.786438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b991960-21ce-4ff1-9641-83e1e20eebcd admin admin] PATH_INFO: `/v3/projects/eae05b5b21524eb58d8e6f47062610d0/users/30096dbf807242d2bd1c82b37c5d98b5/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:06.791067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b991960-21ce-4ff1-9641-83e1e20eebcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae05b5b21524eb58d8e6f47062610d0, user_id=30096dbf807242d2bd1c82b37c5d98b5, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:06.803132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b991960-21ce-4ff1-9641-83e1e20eebcd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:06.825814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 902/1771] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:06 2022] PUT /identity/v3/projects/eae05b5b21524eb58d8e6f47062610d0/users/30096dbf807242d2bd1c82b37c5d98b5/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:06.839206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b685d-7c7e-4108-811b-4189a2f94678 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:06.841779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b685d-7c7e-4108-811b-4189a2f94678 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:06.841779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b685d-7c7e-4108-811b-4189a2f94678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:06.891685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fd0b685d-7c7e-4108-811b-4189a2f94678 None None] MFA Rules not processed for user `30096dbf807242d2bd1c82b37c5d98b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:06.893377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fd0b685d-7c7e-4108-811b-4189a2f94678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:06.938415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 870/1772] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:31:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:06.940239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5668c-c750-45ab-907b-cc24d0dea968 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:06.963774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5668c-c750-45ab-907b-cc24d0dea968 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:06.986431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 903/1773] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:07.023259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b685d-7c7e-4108-811b-4189a2f94678 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:07.054991 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:07.055926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:07.056523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:07.057480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] PATH_INFO: `/v3/users/d1b641a6bdd54994b536958d04962aa3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:07.059130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b641a6bdd54994b536958d04962aa3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:07.063997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:07.152462 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b685d-7c7e-4108-811b-4189a2f94678 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:07.167130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e439a2c-fd4f-48e4-bf57-892bfbb6a86c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:07.168065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e439a2c-fd4f-48e4-bf57-892bfbb6a86c service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:07.168345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e439a2c-fd4f-48e4-bf57-892bfbb6a86c service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:07.168614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e439a2c-fd4f-48e4-bf57-892bfbb6a86c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:07.196702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e439a2c-fd4f-48e4-bf57-892bfbb6a86c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:07.198192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e439a2c-fd4f-48e4-bf57-892bfbb6a86c service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:07.228356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 904/1774] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:07.331112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b641a6bdd54994b536958d04962aa3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:07.333697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b641a6bdd54994b536958d04962aa3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:07.413664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e439a2c-fd4f-48e4-bf57-892bfbb6a86c service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:07.415885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3e439a2c-fd4f-48e4-bf57-892bfbb6a86c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:07.429218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b641a6bdd54994b536958d04962aa3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:07.504109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b641a6bdd54994b536958d04962aa3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:07.536374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-480f9b7b-ae15-4d37-a2a3-8f2dc66bd88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b641a6bdd54994b536958d04962aa3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:07.560323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 871/1775] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:06 2022] DELETE /identity/v3/users/d1b641a6bdd54994b536958d04962aa3 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:07.570460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebbf0-d6ad-4daf-b2da-cc3067660fcd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:07.570930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebbf0-d6ad-4daf-b2da-cc3067660fcd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:07.571375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebbf0-d6ad-4daf-b2da-cc3067660fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:07.639759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1d8ebbf0-d6ad-4daf-b2da-cc3067660fcd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:07.641826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1d8ebbf0-d6ad-4daf-b2da-cc3067660fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:07.803141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 872/1776] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:07.807914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d56ed7-d5e6-4041-8202-b33643b8df42 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:07.809178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d56ed7-d5e6-4041-8202-b33643b8df42 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:07.813507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d56ed7-d5e6-4041-8202-b33643b8df42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:07.881427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e439a2c-fd4f-48e4-bf57-892bfbb6a86c service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:07.912254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a8d56ed7-d5e6-4041-8202-b33643b8df42 None None] MFA Rules not processed for user `0e21910fca40479ca04e7f0418f7dd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:07.918219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a8d56ed7-d5e6-4041-8202-b33643b8df42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:07.967769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:08.066904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb7f27-6b93-4008-bb4e-a058cee9e389 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:08.069352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb7f27-6b93-4008-bb4e-a058cee9e389 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:08.071092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb7f27-6b93-4008-bb4e-a058cee9e389 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:08.071587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb7f27-6b93-4008-bb4e-a058cee9e389 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:08.075793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1ecb7f27-6b93-4008-bb4e-a058cee9e389 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:08.099278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 873/1777] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Sep 1 05:31:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:08.100230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d56ed7-d5e6-4041-8202-b33643b8df42 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:08.101672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a8d56ed7-d5e6-4041-8202-b33643b8df42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:08.175439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d56ed7-d5e6-4041-8202-b33643b8df42 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:08.180304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc026b2-5e11-4a6d-b369-86a51a6231bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:08.181253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc026b2-5e11-4a6d-b369-86a51a6231bf service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:08.181796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc026b2-5e11-4a6d-b369-86a51a6231bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:08.182201 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc026b2-5e11-4a6d-b369-86a51a6231bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:08.185675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5fc026b2-5e11-4a6d-b369-86a51a6231bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:08.206707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb7f27-6b93-4008-bb4e-a058cee9e389 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:08.224491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb7f27-6b93-4008-bb4e-a058cee9e389 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:08.246047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 905/1778] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:31:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 833 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:31:08.249389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb7f27-6b93-4008-bb4e-a058cee9e389 service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:08.257285 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1ecb7f27-6b93-4008-bb4e-a058cee9e389 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:08.277677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb7f27-6b93-4008-bb4e-a058cee9e389 service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:08.297964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc026b2-5e11-4a6d-b369-86a51a6231bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:08.301020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc026b2-5e11-4a6d-b369-86a51a6231bf service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:08.306050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3933fd-f432-4dd6-84dd-2442728be8cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:08.306868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3933fd-f432-4dd6-84dd-2442728be8cb service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:08.307345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3933fd-f432-4dd6-84dd-2442728be8cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:08.307872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3933fd-f432-4dd6-84dd-2442728be8cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:08.326580 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3933fd-f432-4dd6-84dd-2442728be8cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:08.328818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3933fd-f432-4dd6-84dd-2442728be8cb service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:08.357512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 874/1779] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:08.377110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc026b2-5e11-4a6d-b369-86a51a6231bf service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:08.391747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 906/1780] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:08.417889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:08.460226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc026b2-5e11-4a6d-b369-86a51a6231bf service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:08.467011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcb1ec-6287-4540-b366-18eae5bbcf6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:08.467759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcb1ec-6287-4540-b366-18eae5bbcf6c service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:08.468065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcb1ec-6287-4540-b366-18eae5bbcf6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:08.468515 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcb1ec-6287-4540-b366-18eae5bbcf6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:08.482903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcb1ec-6287-4540-b366-18eae5bbcf6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:08.484827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcb1ec-6287-4540-b366-18eae5bbcf6c service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:08.507186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 875/1781] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:10.586207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3933fd-f432-4dd6-84dd-2442728be8cb service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:10.588855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9d3933fd-f432-4dd6-84dd-2442728be8cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:10.640354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3933fd-f432-4dd6-84dd-2442728be8cb service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:10.641336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcb1ec-6287-4540-b366-18eae5bbcf6c service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:10.642972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ecdcb1ec-6287-4540-b366-18eae5bbcf6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:10.654284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:10.656210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:10.658355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:10.659016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] PATH_INFO: `/v3/projects/b2c18a7afbc0429d8f5c1c4b8ab4137a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:10.678199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c18a7afbc0429d8f5c1c4b8ab4137a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:10.681325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:10.716001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcb1ec-6287-4540-b366-18eae5bbcf6c service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:10.758305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:10.767425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:10.770781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:10.771254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] PATH_INFO: `/v3/users/47ae638614b24d8fb3bbc4481c1139bd` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:10.782059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ae638614b24d8fb3bbc4481c1139bd)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:10.790300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:10.794924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:10.899350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c18a7afbc0429d8f5c1c4b8ab4137a', 'request_id': 'req-9629df90-f247-44b6-b9b2-d4a11d638c70'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.005372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c18a7afbc0429d8f5c1c4b8ab4137a', 'request_id': 'req-9629df90-f247-44b6-b9b2-d4a11d638c70'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.019914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c18a7afbc0429d8f5c1c4b8ab4137a', 'request_id': 'req-9629df90-f247-44b6-b9b2-d4a11d638c70'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.062971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] The token cache is being invalidate because project b2c18a7afbc0429d8f5c1c4b8ab4137a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:11.063637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.065740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.071860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 907/1782] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:10 2022] DELETE /identity/v3/projects/b2c18a7afbc0429d8f5c1c4b8ab4137a => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:11.073875 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:11.076965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:11.173275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ae638614b24d8fb3bbc4481c1139bd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.176094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ae638614b24d8fb3bbc4481c1139bd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.303445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ae638614b24d8fb3bbc4481c1139bd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.316224 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629df90-f247-44b6-b9b2-d4a11d638c70 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:11.339509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ae638614b24d8fb3bbc4481c1139bd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.343987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:11.345511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.346869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.347102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] PATH_INFO: `/v3/users/bea4f8dc19ff4502a38d709e468c97b0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.348348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bea4f8dc19ff4502a38d709e468c97b0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:11.351625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:11.355943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5c4a641a-d750-4791-906f-b48472c84d71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ae638614b24d8fb3bbc4481c1139bd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.385180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 876/1783] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:10 2022] DELETE /identity/v3/users/47ae638614b24d8fb3bbc4481c1139bd => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:11.387102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e78152-5de2-4274-8140-92f646ce8d06 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.390106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e78152-5de2-4274-8140-92f646ce8d06 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.395297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e78152-5de2-4274-8140-92f646ce8d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.431905 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-87e78152-5de2-4274-8140-92f646ce8d06 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:11.433368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-87e78152-5de2-4274-8140-92f646ce8d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:11.522653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:11.591485 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 877/1784] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:11.598669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bea4f8dc19ff4502a38d709e468c97b0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.625426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e78152-5de2-4274-8140-92f646ce8d06 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:11.626924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-87e78152-5de2-4274-8140-92f646ce8d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:11.668512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bea4f8dc19ff4502a38d709e468c97b0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.695274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bea4f8dc19ff4502a38d709e468c97b0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.801537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e78152-5de2-4274-8140-92f646ce8d06 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:11.810094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d276ee-9412-49e7-b3b9-c6977bc85bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:11.810913 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d276ee-9412-49e7-b3b9-c6977bc85bfa service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.811193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d276ee-9412-49e7-b3b9-c6977bc85bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.811469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d276ee-9412-49e7-b3b9-c6977bc85bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.822913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bea4f8dc19ff4502a38d709e468c97b0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.824037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-327791c4-06f9-419e-9a65-d80207605ac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bea4f8dc19ff4502a38d709e468c97b0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.858472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d276ee-9412-49e7-b3b9-c6977bc85bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:11.860329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d276ee-9412-49e7-b3b9-c6977bc85bfa service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:11.870228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 908/1785] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:11 2022] DELETE /identity/v3/users/bea4f8dc19ff4502a38d709e468c97b0 => generated 0 bytes in 797 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:11.890092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 878/1786] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:11.971137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f9d2c-8d0e-4aef-ae1c-d76375cf7717 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.971428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f9d2c-8d0e-4aef-ae1c-d76375cf7717 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.971711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f9d2c-8d0e-4aef-ae1c-d76375cf7717 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.973190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 909/1787] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:31:12.057133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7d663-6961-459f-b5a6-018b05d2a968 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.057423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7d663-6961-459f-b5a6-018b05d2a968 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.057862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7d663-6961-459f-b5a6-018b05d2a968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.113750 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2be7d663-6961-459f-b5a6-018b05d2a968 None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:12.115017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2be7d663-6961-459f-b5a6-018b05d2a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:12.305544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 879/1788] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:31:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:12.373755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f9d2c-8d0e-4aef-ae1c-d76375cf7717 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.375972 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9f4f9d2c-8d0e-4aef-ae1c-d76375cf7717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:12.411298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f9d2c-8d0e-4aef-ae1c-d76375cf7717 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.438951 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b75664-50d4-4fb9-bf2d-30ac8d6161e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.440144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b75664-50d4-4fb9-bf2d-30ac8d6161e0 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.440440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b75664-50d4-4fb9-bf2d-30ac8d6161e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.440699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b75664-50d4-4fb9-bf2d-30ac8d6161e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.460596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b75664-50d4-4fb9-bf2d-30ac8d6161e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.462305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b75664-50d4-4fb9-bf2d-30ac8d6161e0 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.479641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 910/1789] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:12.621585 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:13.742803 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7d663-6961-459f-b5a6-018b05d2a968 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.745583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2be7d663-6961-459f-b5a6-018b05d2a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:13.751469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b75664-50d4-4fb9-bf2d-30ac8d6161e0 service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.758481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a5b75664-50d4-4fb9-bf2d-30ac8d6161e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:13.940489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7d663-6961-459f-b5a6-018b05d2a968 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.951885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b75664-50d4-4fb9-bf2d-30ac8d6161e0 service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.954094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.954827 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.955165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.955451 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] PATH_INFO: `/v3/users/c038fb28b3b445669823d1170c6eabf1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.960155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c038fb28b3b445669823d1170c6eabf1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.963092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.970718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.971471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.971864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.972262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] PATH_INFO: `/v3/projects/af62f5cd84804ccfb62ba857bd2629f1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.974388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=af62f5cd84804ccfb62ba857bd2629f1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.978570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:14.207339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af62f5cd84804ccfb62ba857bd2629f1', 'request_id': 'req-240786ca-fe4b-496d-9203-22be836d1d2b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:14.252264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:14.254787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c038fb28b3b445669823d1170c6eabf1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:14.256088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c038fb28b3b445669823d1170c6eabf1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:14.309350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af62f5cd84804ccfb62ba857bd2629f1', 'request_id': 'req-240786ca-fe4b-496d-9203-22be836d1d2b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:14.318603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af62f5cd84804ccfb62ba857bd2629f1', 'request_id': 'req-240786ca-fe4b-496d-9203-22be836d1d2b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:14.374811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] The token cache is being invalidate because project af62f5cd84804ccfb62ba857bd2629f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:14.375981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:14.376778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:14.383545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 911/1790] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:13 2022] DELETE /identity/v3/projects/af62f5cd84804ccfb62ba857bd2629f1 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:14.384442 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:14.390815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:14.400750 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c038fb28b3b445669823d1170c6eabf1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:14.435336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c038fb28b3b445669823d1170c6eabf1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:14.476239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c038fb28b3b445669823d1170c6eabf1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:14.509236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:14.517296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 880/1791] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:13 2022] DELETE /identity/v3/users/c038fb28b3b445669823d1170c6eabf1 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:14.518188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:14.519971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:14.597614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240786ca-fe4b-496d-9203-22be836d1d2b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:14.732606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abadc5b-1b18-4540-bfe1-e34248f833d6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:14.741480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:14.743511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:14.744279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:14.745126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:14.745972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:14.746558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] PATH_INFO: `/v3/projects/9608c76db6a64fc9b9441e4f5b9f162d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:14.751746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:14.753400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9608c76db6a64fc9b9441e4f5b9f162d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:14.754218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] PATH_INFO: `/v3/users/5f33aaa4275c45db99ef83e5b5eb158c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:14.756949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f33aaa4275c45db99ef83e5b5eb158c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:14.758469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:14.760630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:15.067405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f33aaa4275c45db99ef83e5b5eb158c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:15.069493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f33aaa4275c45db99ef83e5b5eb158c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:15.086305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9608c76db6a64fc9b9441e4f5b9f162d', 'request_id': 'req-abf81e27-e566-43e5-bfaa-c64cb023654a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:15.136064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f33aaa4275c45db99ef83e5b5eb158c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:15.161919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f33aaa4275c45db99ef83e5b5eb158c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:15.177276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d3c09f3-82f3-4993-b0d5-8f12c4ff98e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f33aaa4275c45db99ef83e5b5eb158c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:15.189286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 881/1792] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:14 2022] DELETE /identity/v3/users/5f33aaa4275c45db99ef83e5b5eb158c => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:15.195653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919d511-6412-45c3-afac-3403525a11ce None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:15.197846 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919d511-6412-45c3-afac-3403525a11ce None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:15.201832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919d511-6412-45c3-afac-3403525a11ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:15.204573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9608c76db6a64fc9b9441e4f5b9f162d', 'request_id': 'req-abf81e27-e566-43e5-bfaa-c64cb023654a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:15.211636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9608c76db6a64fc9b9441e4f5b9f162d', 'request_id': 'req-abf81e27-e566-43e5-bfaa-c64cb023654a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:15.249582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8919d511-6412-45c3-afac-3403525a11ce None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:15.250576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8919d511-6412-45c3-afac-3403525a11ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:15.259190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] The token cache is being invalidate because project 9608c76db6a64fc9b9441e4f5b9f162d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:15.259502 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:15.260167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-abf81e27-e566-43e5-bfaa-c64cb023654a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:15.265125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 912/1793] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:14 2022] DELETE /identity/v3/projects/9608c76db6a64fc9b9441e4f5b9f162d => generated 0 bytes in 878 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:15.267273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc95672-241f-4f24-8b43-0c85b3e3e7b1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:15.267474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc95672-241f-4f24-8b43-0c85b3e3e7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:15.267750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc95672-241f-4f24-8b43-0c85b3e3e7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:15.274101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:15.303503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:15.309037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-abc95672-241f-4f24-8b43-0c85b3e3e7b1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:15.312540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-abc95672-241f-4f24-8b43-0c85b3e3e7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:15.428821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 882/1794] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:15.463711 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919d511-6412-45c3-afac-3403525a11ce None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:15.465541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8919d511-6412-45c3-afac-3403525a11ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:15.538175 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 913/1795] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:15.540161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2d15a-d6d6-43bc-b9b4-79970e6db021 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:15.540461 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2d15a-d6d6-43bc-b9b4-79970e6db021 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:15.540792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2d15a-d6d6-43bc-b9b4-79970e6db021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:15.611040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f8c2d15a-d6d6-43bc-b9b4-79970e6db021 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:15.621571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f8c2d15a-d6d6-43bc-b9b4-79970e6db021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:15.629686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919d511-6412-45c3-afac-3403525a11ce None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:15.639478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 914/1796] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:15.639898 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2d15a-d6d6-43bc-b9b4-79970e6db021 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:15.645690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b9587-dc8f-4a40-818e-463fc8970cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:15.646372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b9587-dc8f-4a40-818e-463fc8970cfd service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:15.646642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b9587-dc8f-4a40-818e-463fc8970cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:15.647429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b9587-dc8f-4a40-818e-463fc8970cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:15.659969 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2d15a-d6d6-43bc-b9b4-79970e6db021 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:15.668390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50f383-4259-43f5-8a19-552a1360a940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:15.668390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50f383-4259-43f5-8a19-552a1360a940 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:15.671054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50f383-4259-43f5-8a19-552a1360a940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:15.671992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50f383-4259-43f5-8a19-552a1360a940 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:15.673303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50f383-4259-43f5-8a19-552a1360a940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:15.673428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b9587-dc8f-4a40-818e-463fc8970cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:15.673928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50f383-4259-43f5-8a19-552a1360a940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:15.674859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b9587-dc8f-4a40-818e-463fc8970cfd service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:15.676074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50f383-4259-43f5-8a19-552a1360a940 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:15.747873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 883/1797] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:31:15.754131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 915/1798] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:31:15.755178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50f383-4259-43f5-8a19-552a1360a940 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:15.782215 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50f383-4259-43f5-8a19-552a1360a940 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:15.798368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448c7c8-f3b1-4231-8831-d45926b44d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:15.798998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448c7c8-f3b1-4231-8831-d45926b44d09 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:15.799421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448c7c8-f3b1-4231-8831-d45926b44d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:15.799709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448c7c8-f3b1-4231-8831-d45926b44d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:15.800346 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448c7c8-f3b1-4231-8831-d45926b44d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:15.802289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448c7c8-f3b1-4231-8831-d45926b44d09 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:15.928810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 916/1799] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:15 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:15.932772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b9587-dc8f-4a40-818e-463fc8970cfd service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:16.011255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b9587-dc8f-4a40-818e-463fc8970cfd service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:16.231575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9b25d-7f7c-471c-95d1-1b71d73c5558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:16.233257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9b25d-7f7c-471c-95d1-1b71d73c5558 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:16.233834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9b25d-7f7c-471c-95d1-1b71d73c5558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:16.234500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9b25d-7f7c-471c-95d1-1b71d73c5558 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:16.235608 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9b25d-7f7c-471c-95d1-1b71d73c5558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:16.239841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9b25d-7f7c-471c-95d1-1b71d73c5558 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:16.269030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:16.363710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 884/1800] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:15 2022] POST /identity/v3/users => generated 283 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:16.368103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448c7c8-f3b1-4231-8831-d45926b44d09 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:16.399360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448c7c8-f3b1-4231-8831-d45926b44d09 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:16.406637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99a36f-cb7e-4a51-aee2-8f8dd0c52cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:16.407556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99a36f-cb7e-4a51-aee2-8f8dd0c52cbb admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:16.407991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99a36f-cb7e-4a51-aee2-8f8dd0c52cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:16.408582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99a36f-cb7e-4a51-aee2-8f8dd0c52cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:16.420930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99a36f-cb7e-4a51-aee2-8f8dd0c52cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:16.431558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99a36f-cb7e-4a51-aee2-8f8dd0c52cbb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:16.439411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 917/1801] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:16.443293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9b25d-7f7c-471c-95d1-1b71d73c5558 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:16.462721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9b25d-7f7c-471c-95d1-1b71d73c5558 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:16.469622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c7605-802f-4cae-b36d-d4f21c7f9582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:16.470250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c7605-802f-4cae-b36d-d4f21c7f9582 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:16.470524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c7605-802f-4cae-b36d-d4f21c7f9582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:16.470879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c7605-802f-4cae-b36d-d4f21c7f9582 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:16.471490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c7605-802f-4cae-b36d-d4f21c7f9582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:16.473901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c7605-802f-4cae-b36d-d4f21c7f9582 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:16.509828 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 885/1802] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:16.509946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99a36f-cb7e-4a51-aee2-8f8dd0c52cbb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:16.524076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99a36f-cb7e-4a51-aee2-8f8dd0c52cbb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:16.552632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1b6d4-ed07-42e3-b252-24fe696a4ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:16.554209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1b6d4-ed07-42e3-b252-24fe696a4ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:16.554700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1b6d4-ed07-42e3-b252-24fe696a4ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:16.555444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1b6d4-ed07-42e3-b252-24fe696a4ee6 admin admin] PATH_INFO: `/v3/projects/aa8f493817594828931b4c8fe7d3cb93/users/9c90b89c710840c9951722a8432d620e/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:16.595107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1b6d4-ed07-42e3-b252-24fe696a4ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8f493817594828931b4c8fe7d3cb93, user_id=9c90b89c710840c9951722a8432d620e, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:16.600256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1b6d4-ed07-42e3-b252-24fe696a4ee6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:16.693327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 918/1803] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:16 2022] PUT /identity/v3/projects/aa8f493817594828931b4c8fe7d3cb93/users/9c90b89c710840c9951722a8432d620e/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:16.705493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47871d5-9c73-45c2-8fe4-2e4689cffa0d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:16.705786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47871d5-9c73-45c2-8fe4-2e4689cffa0d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:16.706069 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47871d5-9c73-45c2-8fe4-2e4689cffa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:16.773051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:16.777121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c47871d5-9c73-45c2-8fe4-2e4689cffa0d None None] MFA Rules not processed for user `9c90b89c710840c9951722a8432d620e`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:16.782681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c47871d5-9c73-45c2-8fe4-2e4689cffa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:16.973718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 886/1804] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:16.977997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1b6d4-ed07-42e3-b252-24fe696a4ee6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:17.001149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47871d5-9c73-45c2-8fe4-2e4689cffa0d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:17.003180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c47871d5-9c73-45c2-8fe4-2e4689cffa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:17.152677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1b6d4-ed07-42e3-b252-24fe696a4ee6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:17.166400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fca04-db3c-428d-b7c0-bb232aba48e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:17.167423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fca04-db3c-428d-b7c0-bb232aba48e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:17.167920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fca04-db3c-428d-b7c0-bb232aba48e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:17.168407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fca04-db3c-428d-b7c0-bb232aba48e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:17.169409 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fca04-db3c-428d-b7c0-bb232aba48e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:17.172670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fca04-db3c-428d-b7c0-bb232aba48e1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:17.189531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47871d5-9c73-45c2-8fe4-2e4689cffa0d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:17.217847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:17.217847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:17.217847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:17.217847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] PATH_INFO: `/v3/projects/7393ba3c406b4e0096584ce1b7234502` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:17.230036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7393ba3c406b4e0096584ce1b7234502)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:17.234613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:17.285316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 919/1805] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:16 2022] POST /identity/v3/projects => generated 363 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:17.287614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fca04-db3c-428d-b7c0-bb232aba48e1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:17.413973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7393ba3c406b4e0096584ce1b7234502', 'request_id': 'req-c49fe247-d496-40a8-ad37-8c805a7ad1f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:17.458819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:17.466893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fca04-db3c-428d-b7c0-bb232aba48e1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:17.545608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62aa02-ac34-41c6-b894-cbe07f31a9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:17.546734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62aa02-ac34-41c6-b894-cbe07f31a9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:17.547392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62aa02-ac34-41c6-b894-cbe07f31a9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:17.547888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62aa02-ac34-41c6-b894-cbe07f31a9d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:17.548737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62aa02-ac34-41c6-b894-cbe07f31a9d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:17.551390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62aa02-ac34-41c6-b894-cbe07f31a9d2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:17.554692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7393ba3c406b4e0096584ce1b7234502', 'request_id': 'req-c49fe247-d496-40a8-ad37-8c805a7ad1f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:17.565580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7393ba3c406b4e0096584ce1b7234502', 'request_id': 'req-c49fe247-d496-40a8-ad37-8c805a7ad1f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:17.581380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] The token cache is being invalidate because project 7393ba3c406b4e0096584ce1b7234502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:17.582037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:17.583025 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:17.585321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 887/1806] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:17 2022] DELETE /identity/v3/projects/7393ba3c406b4e0096584ce1b7234502 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:17.586452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:17.588743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:17.602343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fe247-d496-40a8-ad37-8c805a7ad1f4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:17.608266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:17.610048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:17.610526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:17.611085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] PATH_INFO: `/v3/projects/bf01bf726fbc4d70b243695cb4d92bdf` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:17.612520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf01bf726fbc4d70b243695cb4d92bdf)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:17.615986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:17.685962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 920/1807] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:17 2022] POST /identity/v3/users => generated 288 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:17.685962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62aa02-ac34-41c6-b894-cbe07f31a9d2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:17.685962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bd62aa02-ac34-41c6-b894-cbe07f31a9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:17.707135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62aa02-ac34-41c6-b894-cbe07f31a9d2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:17.722467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:17.728686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:17.728686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:17.728686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] PATH_INFO: `/v3/users/30096dbf807242d2bd1c82b37c5d98b5` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:17.728686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=30096dbf807242d2bd1c82b37c5d98b5)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:17.736202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:17.812432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf01bf726fbc4d70b243695cb4d92bdf', 'request_id': 'req-72b4af77-ac07-4212-95bf-3a05d30dc9d5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:17.955354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf01bf726fbc4d70b243695cb4d92bdf', 'request_id': 'req-72b4af77-ac07-4212-95bf-3a05d30dc9d5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:17.979098 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf01bf726fbc4d70b243695cb4d92bdf', 'request_id': 'req-72b4af77-ac07-4212-95bf-3a05d30dc9d5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.017909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] The token cache is being invalidate because project bf01bf726fbc4d70b243695cb4d92bdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:18.018547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.020667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.023773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 888/1808] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:17 2022] DELETE /identity/v3/projects/bf01bf726fbc4d70b243695cb4d92bdf => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:18.026194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:18.033655 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:18.115315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30096dbf807242d2bd1c82b37c5d98b5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.170266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30096dbf807242d2bd1c82b37c5d98b5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.187165 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30096dbf807242d2bd1c82b37c5d98b5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.220746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4af77-ac07-4212-95bf-3a05d30dc9d5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:18.255064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c471d-8da7-40e1-965e-cebba481249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:18.255775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c471d-8da7-40e1-965e-cebba481249e admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:18.256107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c471d-8da7-40e1-965e-cebba481249e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:18.256383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c471d-8da7-40e1-965e-cebba481249e admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:18.257161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c471d-8da7-40e1-965e-cebba481249e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:18.261141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c471d-8da7-40e1-965e-cebba481249e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:18.278881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 889/1809] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:18.280879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c471d-8da7-40e1-965e-cebba481249e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:18.283340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c76c471d-8da7-40e1-965e-cebba481249e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:18.297591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c471d-8da7-40e1-965e-cebba481249e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:18.303925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:18.304920 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:18.305271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:18.305640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] PATH_INFO: `/v3/users/b8fb449e8d3349c9a0d5bf5f55d2c992` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:18.307464 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8fb449e8d3349c9a0d5bf5f55d2c992)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:18.311619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:18.313152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30096dbf807242d2bd1c82b37c5d98b5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.314034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5e8e48ae-8cf7-4858-8f85-788b02370b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30096dbf807242d2bd1c82b37c5d98b5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.328377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 921/1810] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:17 2022] DELETE /identity/v3/users/30096dbf807242d2bd1c82b37c5d98b5 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:18.335340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726c96d-7719-44e2-a4a3-4ee0408c8307 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:18.335880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726c96d-7719-44e2-a4a3-4ee0408c8307 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:18.336560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726c96d-7719-44e2-a4a3-4ee0408c8307 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:18.338529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 922/1811] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:31:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:31:18.339240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726c96d-7719-44e2-a4a3-4ee0408c8307 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:18.342564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0726c96d-7719-44e2-a4a3-4ee0408c8307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:18.472412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726c96d-7719-44e2-a4a3-4ee0408c8307 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:18.506913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b4041-452c-4131-9703-a6f46d8c04ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:18.510516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4041-452c-4131-9703-a6f46d8c04ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:18.510952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4041-452c-4131-9703-a6f46d8c04ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:18.511563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4041-452c-4131-9703-a6f46d8c04ac admin admin] PATH_INFO: `/v3/projects/59866ccbc2a04bc7b471dd74f69cc343/users/3b29c591b19e47c8be6a86b315bf41a9/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:18.537782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8fb449e8d3349c9a0d5bf5f55d2c992'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.540611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8fb449e8d3349c9a0d5bf5f55d2c992'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.553039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:18.626339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8fb449e8d3349c9a0d5bf5f55d2c992'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.644882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b4041-452c-4131-9703-a6f46d8c04ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=59866ccbc2a04bc7b471dd74f69cc343, user_id=3b29c591b19e47c8be6a86b315bf41a9, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:18.649665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8fb449e8d3349c9a0d5bf5f55d2c992'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.652830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b4041-452c-4131-9703-a6f46d8c04ac admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:18.691891 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72ac079b-819e-4853-acf8-631ba0ed1304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8fb449e8d3349c9a0d5bf5f55d2c992'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:18.718258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 890/1812] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:18 2022] DELETE /identity/v3/users/b8fb449e8d3349c9a0d5bf5f55d2c992 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:18.726677 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d442a3-0fe5-437f-869a-cd72deee9c4e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:18.730668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d442a3-0fe5-437f-869a-cd72deee9c4e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:18.730668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d442a3-0fe5-437f-869a-cd72deee9c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:18.761194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 923/1813] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:18 2022] PUT /identity/v3/projects/59866ccbc2a04bc7b471dd74f69cc343/users/3b29c591b19e47c8be6a86b315bf41a9/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:18.770205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b4041-452c-4131-9703-a6f46d8c04ac admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:18.776509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1c0b4041-452c-4131-9703-a6f46d8c04ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:18.814873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f3d442a3-0fe5-437f-869a-cd72deee9c4e None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:18.825221 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f3d442a3-0fe5-437f-869a-cd72deee9c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:18.984341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b4041-452c-4131-9703-a6f46d8c04ac admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:19.007019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73094da3-f0cc-4574-937f-0378a76e92ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:19.011179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73094da3-f0cc-4574-937f-0378a76e92ed admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:19.011520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73094da3-f0cc-4574-937f-0378a76e92ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:19.011795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73094da3-f0cc-4574-937f-0378a76e92ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:19.014689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73094da3-f0cc-4574-937f-0378a76e92ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:19.018546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73094da3-f0cc-4574-937f-0378a76e92ed admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:19.059244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 924/1814] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:19.062930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73094da3-f0cc-4574-937f-0378a76e92ed admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:19.074246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 891/1815] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:31:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:19.088122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73094da3-f0cc-4574-937f-0378a76e92ed admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:19.108328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b198e-63ef-4fa1-bfda-0c5e25cd0b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:19.109037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b198e-63ef-4fa1-bfda-0c5e25cd0b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:19.109372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b198e-63ef-4fa1-bfda-0c5e25cd0b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:19.109698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b198e-63ef-4fa1-bfda-0c5e25cd0b4c admin admin] PATH_INFO: `/v3/projects/59866ccbc2a04bc7b471dd74f69cc343/users/3b29c591b19e47c8be6a86b315bf41a9/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:19.116891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b198e-63ef-4fa1-bfda-0c5e25cd0b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59866ccbc2a04bc7b471dd74f69cc343, user_id=3b29c591b19e47c8be6a86b315bf41a9, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:19.123271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b198e-63ef-4fa1-bfda-0c5e25cd0b4c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:19.143823 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 925/1816] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:19 2022] PUT /identity/v3/projects/59866ccbc2a04bc7b471dd74f69cc343/users/3b29c591b19e47c8be6a86b315bf41a9/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:19.155853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a067a-c6bd-41b3-ae0f-76e157be7ff1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:19.156488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a067a-c6bd-41b3-ae0f-76e157be7ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:19.157115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a067a-c6bd-41b3-ae0f-76e157be7ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:19.205426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1a5a067a-c6bd-41b3-ae0f-76e157be7ff1 None None] MFA Rules not processed for user `3b29c591b19e47c8be6a86b315bf41a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:19.208046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1a5a067a-c6bd-41b3-ae0f-76e157be7ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:19.278173 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b198e-63ef-4fa1-bfda-0c5e25cd0b4c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:19.282608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c23b198e-63ef-4fa1-bfda-0c5e25cd0b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:19.406230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 892/1817] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:19.416839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a067a-c6bd-41b3-ae0f-76e157be7ff1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:19.418687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1a5a067a-c6bd-41b3-ae0f-76e157be7ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:19.446465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b198e-63ef-4fa1-bfda-0c5e25cd0b4c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:19.464824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d9221-e96e-409d-8572-c31942cbc92e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:19.465484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d9221-e96e-409d-8572-c31942cbc92e service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:19.465778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d9221-e96e-409d-8572-c31942cbc92e service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:19.466059 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d9221-e96e-409d-8572-c31942cbc92e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:19.479234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d9221-e96e-409d-8572-c31942cbc92e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:19.481517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d9221-e96e-409d-8572-c31942cbc92e service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:19.500187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 926/1818] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:19.615239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a067a-c6bd-41b3-ae0f-76e157be7ff1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:19.624901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1edb7-a8dd-46a3-ae27-19403dc469d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:19.625899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1edb7-a8dd-46a3-ae27-19403dc469d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:19.627090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1edb7-a8dd-46a3-ae27-19403dc469d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:19.627566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1edb7-a8dd-46a3-ae27-19403dc469d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:19.644578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1edb7-a8dd-46a3-ae27-19403dc469d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:19.646336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1edb7-a8dd-46a3-ae27-19403dc469d7 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:19.719415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 893/1819] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:19.852844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:19.949062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d9221-e96e-409d-8572-c31942cbc92e service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:19.950834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-be1d9221-e96e-409d-8572-c31942cbc92e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:20.071370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1edb7-a8dd-46a3-ae27-19403dc469d7 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:20.087517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1edb7-a8dd-46a3-ae27-19403dc469d7 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:20.103705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe072f-135d-430c-8b38-749eb057ae46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:20.104604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe072f-135d-430c-8b38-749eb057ae46 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.105058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe072f-135d-430c-8b38-749eb057ae46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.105831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe072f-135d-430c-8b38-749eb057ae46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.110689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cebe072f-135d-430c-8b38-749eb057ae46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:20.149174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d9221-e96e-409d-8572-c31942cbc92e service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:20.200921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:20.202247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.204383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.205835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] PATH_INFO: `/v3/projects/eae05b5b21524eb58d8e6f47062610d0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.207393 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae05b5b21524eb58d8e6f47062610d0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:20.211448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:20.270075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe072f-135d-430c-8b38-749eb057ae46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:20.272116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe072f-135d-430c-8b38-749eb057ae46 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:20.285477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 894/1820] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:31:20.287168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe072f-135d-430c-8b38-749eb057ae46 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:20.290436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cebe072f-135d-430c-8b38-749eb057ae46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:20.307144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe072f-135d-430c-8b38-749eb057ae46 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:20.313595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:20.314478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.314761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.315028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] PATH_INFO: `/v3/projects/1195c2b2b55546bca8a0e441a73d5210` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.316875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1195c2b2b55546bca8a0e441a73d5210)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:20.319244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:20.409221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae05b5b21524eb58d8e6f47062610d0', 'request_id': 'req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.486397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae05b5b21524eb58d8e6f47062610d0', 'request_id': 'req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.497384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae05b5b21524eb58d8e6f47062610d0', 'request_id': 'req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.523342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1195c2b2b55546bca8a0e441a73d5210', 'request_id': 'req-70e54478-010a-4d5c-80bb-620535b41684'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.525402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] The token cache is being invalidate because project eae05b5b21524eb58d8e6f47062610d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:20.525850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.526615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-77800e4f-bb76-4df0-9ae4-f44bfd2ca7c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.528527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 927/1821] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:19 2022] DELETE /identity/v3/projects/eae05b5b21524eb58d8e6f47062610d0 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:20.534427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaca5c-0d7c-4665-ba62-06e1ae0c22a0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.534427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaca5c-0d7c-4665-ba62-06e1ae0c22a0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.534637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaca5c-0d7c-4665-ba62-06e1ae0c22a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.573820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ffaaca5c-0d7c-4665-ba62-06e1ae0c22a0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:20.578897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ffaaca5c-0d7c-4665-ba62-06e1ae0c22a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:20.658629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1195c2b2b55546bca8a0e441a73d5210', 'request_id': 'req-70e54478-010a-4d5c-80bb-620535b41684'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.671063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1195c2b2b55546bca8a0e441a73d5210', 'request_id': 'req-70e54478-010a-4d5c-80bb-620535b41684'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.685653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:20.706946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 928/1822] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:20.717118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eda633-d5b1-442f-97f5-ea4d23658736 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.718250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eda633-d5b1-442f-97f5-ea4d23658736 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.718250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eda633-d5b1-442f-97f5-ea4d23658736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.718526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] The token cache is being invalidate because project 1195c2b2b55546bca8a0e441a73d5210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:20.718934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.719760 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70e54478-010a-4d5c-80bb-620535b41684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.721906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 895/1823] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:20 2022] DELETE /identity/v3/projects/1195c2b2b55546bca8a0e441a73d5210 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:20.728172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faf94b-4b58-433f-9cba-f43def813cf1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.732143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faf94b-4b58-433f-9cba-f43def813cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.732143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faf94b-4b58-433f-9cba-f43def813cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.766460 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-64eda633-d5b1-442f-97f5-ea4d23658736 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:20.770768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-64eda633-d5b1-442f-97f5-ea4d23658736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:20.784877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-10faf94b-4b58-433f-9cba-f43def813cf1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:20.785707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-10faf94b-4b58-433f-9cba-f43def813cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:20.802254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 929/1824] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:20.805745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eda633-d5b1-442f-97f5-ea4d23658736 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:20.806987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 896/1825] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:20.817161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eda633-d5b1-442f-97f5-ea4d23658736 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:20.826577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95f1bb-731f-42f2-80b2-588ce6bebb94 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.826975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95f1bb-731f-42f2-80b2-588ce6bebb94 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.827191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95f1bb-731f-42f2-80b2-588ce6bebb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.849650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331d2fa-aa80-4ec0-8c33-c6285b4ea989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:20.851750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331d2fa-aa80-4ec0-8c33-c6285b4ea989 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.852036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331d2fa-aa80-4ec0-8c33-c6285b4ea989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.853182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331d2fa-aa80-4ec0-8c33-c6285b4ea989 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.862314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331d2fa-aa80-4ec0-8c33-c6285b4ea989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:20.867153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331d2fa-aa80-4ec0-8c33-c6285b4ea989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:20.867153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331d2fa-aa80-4ec0-8c33-c6285b4ea989 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:20.884273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cd95f1bb-731f-42f2-80b2-588ce6bebb94 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:20.885856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cd95f1bb-731f-42f2-80b2-588ce6bebb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:20.893607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 930/1826] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:31:20.899515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331d2fa-aa80-4ec0-8c33-c6285b4ea989 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:20.899695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 897/1827] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:20.920076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331d2fa-aa80-4ec0-8c33-c6285b4ea989 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:20.920219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95f1bb-731f-42f2-80b2-588ce6bebb94 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:20.923191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f78ff-2070-4f33-aff4-54c95c44a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:20.924960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f78ff-2070-4f33-aff4-54c95c44a604 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.925252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f78ff-2070-4f33-aff4-54c95c44a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.925740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f78ff-2070-4f33-aff4-54c95c44a604 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.926401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f78ff-2070-4f33-aff4-54c95c44a604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:20.929236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f78ff-2070-4f33-aff4-54c95c44a604 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:20.939340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95f1bb-731f-42f2-80b2-588ce6bebb94 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:20.955419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8208c1d-3a34-433f-ad83-d9c5ac523431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:20.956043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208c1d-3a34-433f-ad83-d9c5ac523431 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.956315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208c1d-3a34-433f-ad83-d9c5ac523431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.957261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208c1d-3a34-433f-ad83-d9c5ac523431 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.957261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8208c1d-3a34-433f-ad83-d9c5ac523431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:20.957662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8208c1d-3a34-433f-ad83-d9c5ac523431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:20.977183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8208c1d-3a34-433f-ad83-d9c5ac523431 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.006265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 898/1828] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:31:21.012666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8208c1d-3a34-433f-ad83-d9c5ac523431 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.042350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8208c1d-3a34-433f-ad83-d9c5ac523431 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.048759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88443907-4c29-41bb-86fd-30291696bcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.049398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88443907-4c29-41bb-86fd-30291696bcbb admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.049727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88443907-4c29-41bb-86fd-30291696bcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.050797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88443907-4c29-41bb-86fd-30291696bcbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.051510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88443907-4c29-41bb-86fd-30291696bcbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.053600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88443907-4c29-41bb-86fd-30291696bcbb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.072697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 931/1829] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:20 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:21.074768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f78ff-2070-4f33-aff4-54c95c44a604 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.131859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:21.187347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f78ff-2070-4f33-aff4-54c95c44a604 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.196384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 899/1830] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:21 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:21.197401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88443907-4c29-41bb-86fd-30291696bcbb admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.197567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6774d-4cdf-46cb-bc46-4b614f526765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.198147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6774d-4cdf-46cb-bc46-4b614f526765 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.198446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6774d-4cdf-46cb-bc46-4b614f526765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.198748 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6774d-4cdf-46cb-bc46-4b614f526765 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.199402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6774d-4cdf-46cb-bc46-4b614f526765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.201931 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6774d-4cdf-46cb-bc46-4b614f526765 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.297074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 932/1831] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:21 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:21.298737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6774d-4cdf-46cb-bc46-4b614f526765 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.313181 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6774d-4cdf-46cb-bc46-4b614f526765 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.319713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53802c7b-64ce-434d-86c5-272d6540cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.320646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53802c7b-64ce-434d-86c5-272d6540cfce admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.326654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53802c7b-64ce-434d-86c5-272d6540cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.326654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53802c7b-64ce-434d-86c5-272d6540cfce admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.326654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53802c7b-64ce-434d-86c5-272d6540cfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.326654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53802c7b-64ce-434d-86c5-272d6540cfce admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.337835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88443907-4c29-41bb-86fd-30291696bcbb admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.346840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 933/1832] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:21.348312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53802c7b-64ce-434d-86c5-272d6540cfce admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.349357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9b38b-0a4f-4944-a2a8-84737e28243f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.351753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9b38b-0a4f-4944-a2a8-84737e28243f admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.355027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9b38b-0a4f-4944-a2a8-84737e28243f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.355027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9b38b-0a4f-4944-a2a8-84737e28243f admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.355027 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9b38b-0a4f-4944-a2a8-84737e28243f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.356383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9b38b-0a4f-4944-a2a8-84737e28243f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.362885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53802c7b-64ce-434d-86c5-272d6540cfce admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.369239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddece94-dfd4-45df-80a2-5710ce2817c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.370823 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddece94-dfd4-45df-80a2-5710ce2817c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.371258 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddece94-dfd4-45df-80a2-5710ce2817c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.371942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddece94-dfd4-45df-80a2-5710ce2817c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.373559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddece94-dfd4-45df-80a2-5710ce2817c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.375451 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddece94-dfd4-45df-80a2-5710ce2817c6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.390098 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 934/1833] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:21.394428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddece94-dfd4-45df-80a2-5710ce2817c6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.410706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddece94-dfd4-45df-80a2-5710ce2817c6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.418497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738d7a2-5d0b-4221-b073-a03a30f15fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.418497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738d7a2-5d0b-4221-b073-a03a30f15fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.418497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738d7a2-5d0b-4221-b073-a03a30f15fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.418497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738d7a2-5d0b-4221-b073-a03a30f15fbb admin admin] PATH_INFO: `/v3/projects/d9a5e4b1c7d649a1bd9de83bf3e81a9d/users/816e74d815c5466c8150a03b7150dce3/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.440288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738d7a2-5d0b-4221-b073-a03a30f15fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a5e4b1c7d649a1bd9de83bf3e81a9d, user_id=816e74d815c5466c8150a03b7150dce3, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.443893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738d7a2-5d0b-4221-b073-a03a30f15fbb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.487416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 900/1834] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:21 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:21.488787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9b38b-0a4f-4944-a2a8-84737e28243f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.515541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9b38b-0a4f-4944-a2a8-84737e28243f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.523364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6819e7a-a1ad-4069-84cf-0de28ea38dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.524105 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6819e7a-a1ad-4069-84cf-0de28ea38dcf admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.524876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6819e7a-a1ad-4069-84cf-0de28ea38dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.529039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6819e7a-a1ad-4069-84cf-0de28ea38dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.530813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6819e7a-a1ad-4069-84cf-0de28ea38dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.533303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6819e7a-a1ad-4069-84cf-0de28ea38dcf admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.548957 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 935/1835] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:21 2022] PUT /identity/v3/projects/d9a5e4b1c7d649a1bd9de83bf3e81a9d/users/816e74d815c5466c8150a03b7150dce3/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:21.553649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 901/1836] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:21.558072 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738d7a2-5d0b-4221-b073-a03a30f15fbb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.559865 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ccc1-8933-4a92-937c-2d9c612891f3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.560915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ccc1-8933-4a92-937c-2d9c612891f3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.561260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ccc1-8933-4a92-937c-2d9c612891f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.594459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6ff6ccc1-8933-4a92-937c-2d9c612891f3 None None] MFA Rules not processed for user `816e74d815c5466c8150a03b7150dce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:21.596004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6ff6ccc1-8933-4a92-937c-2d9c612891f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:21.680590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738d7a2-5d0b-4221-b073-a03a30f15fbb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.686128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e8340-41f1-46b5-84c3-ce5e458cce99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.687106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8340-41f1-46b5-84c3-ce5e458cce99 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.687496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8340-41f1-46b5-84c3-ce5e458cce99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.687911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8340-41f1-46b5-84c3-ce5e458cce99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.688635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e8340-41f1-46b5-84c3-ce5e458cce99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.690578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e8340-41f1-46b5-84c3-ce5e458cce99 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.690978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 902/1837] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:21.694956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6ccc1-8933-4a92-937c-2d9c612891f3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.705614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 936/1838] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:21.709781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6ccc1-8933-4a92-937c-2d9c612891f3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.712726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e8340-41f1-46b5-84c3-ce5e458cce99 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.715382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c917a4-ac04-41f5-bd55-e14cb8411d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.717569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c917a4-ac04-41f5-bd55-e14cb8411d8e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.717946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c917a4-ac04-41f5-bd55-e14cb8411d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.718228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c917a4-ac04-41f5-bd55-e14cb8411d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.718887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c917a4-ac04-41f5-bd55-e14cb8411d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.721030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c917a4-ac04-41f5-bd55-e14cb8411d8e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.725668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e8340-41f1-46b5-84c3-ce5e458cce99 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.739612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9f984-18d8-4535-9b68-8c06e3b8dc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.739612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9f984-18d8-4535-9b68-8c06e3b8dc1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.739612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9f984-18d8-4535-9b68-8c06e3b8dc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.739612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9f984-18d8-4535-9b68-8c06e3b8dc1b admin admin] PATH_INFO: `/v3/projects/801ea44868d1482682628db4ae80f3ed/users/e6e8fc72d34e430b92930e3a40028861/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.752804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9f984-18d8-4535-9b68-8c06e3b8dc1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=801ea44868d1482682628db4ae80f3ed, user_id=e6e8fc72d34e430b92930e3a40028861, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.755484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9f984-18d8-4535-9b68-8c06e3b8dc1b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.827871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 903/1839] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:21 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:21.831857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c917a4-ac04-41f5-bd55-e14cb8411d8e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.861502 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 937/1840] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:21 2022] PUT /identity/v3/projects/801ea44868d1482682628db4ae80f3ed/users/e6e8fc72d34e430b92930e3a40028861/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:21.878716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74217c-130c-4045-bc13-0e23da72c5be None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.879144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74217c-130c-4045-bc13-0e23da72c5be None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.879531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74217c-130c-4045-bc13-0e23da72c5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.897870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c917a4-ac04-41f5-bd55-e14cb8411d8e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.908960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ca74217c-130c-4045-bc13-0e23da72c5be None None] MFA Rules not processed for user `e6e8fc72d34e430b92930e3a40028861`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:21.910211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ca74217c-130c-4045-bc13-0e23da72c5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:21.955794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c347e66-6439-4b03-b24f-c3a6a76539f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.956437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c347e66-6439-4b03-b24f-c3a6a76539f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.956712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c347e66-6439-4b03-b24f-c3a6a76539f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.957007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c347e66-6439-4b03-b24f-c3a6a76539f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.957817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c347e66-6439-4b03-b24f-c3a6a76539f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.959816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c347e66-6439-4b03-b24f-c3a6a76539f3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.974807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 938/1841] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:21.987141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74217c-130c-4045-bc13-0e23da72c5be None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.988682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ca74217c-130c-4045-bc13-0e23da72c5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:22.054700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74217c-130c-4045-bc13-0e23da72c5be None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.058387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f156-32eb-4ab3-af3b-b6b6df72313e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.058996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f156-32eb-4ab3-af3b-b6b6df72313e service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.059272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f156-32eb-4ab3-af3b-b6b6df72313e service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.059824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f156-32eb-4ab3-af3b-b6b6df72313e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.070124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834f156-32eb-4ab3-af3b-b6b6df72313e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.071551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834f156-32eb-4ab3-af3b-b6b6df72313e service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.076210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 904/1842] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:21 2022] POST /identity/v3/users => generated 296 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:22.080350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c347e66-6439-4b03-b24f-c3a6a76539f3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.085333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 939/1843] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:22.091114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c347e66-6439-4b03-b24f-c3a6a76539f3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.097649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293437be-13d7-411b-aeaa-806690d3f085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.098430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293437be-13d7-411b-aeaa-806690d3f085 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.098882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293437be-13d7-411b-aeaa-806690d3f085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.099289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293437be-13d7-411b-aeaa-806690d3f085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.100090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293437be-13d7-411b-aeaa-806690d3f085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.105642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293437be-13d7-411b-aeaa-806690d3f085 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.116088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 905/1844] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:22.120451 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f156-32eb-4ab3-af3b-b6b6df72313e service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.132675 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f156-32eb-4ab3-af3b-b6b6df72313e service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.138732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d150bd-fea6-44f5-b40f-43134bd88cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.139530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d150bd-fea6-44f5-b40f-43134bd88cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.139879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d150bd-fea6-44f5-b40f-43134bd88cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.140223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d150bd-fea6-44f5-b40f-43134bd88cdd admin admin] PATH_INFO: `/v3/projects/cb67074d843a44ef857b0d706303e477/users/e0aff2c2766e41e39ccba49697b2ecb7/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.162630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d150bd-fea6-44f5-b40f-43134bd88cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb67074d843a44ef857b0d706303e477, user_id=e0aff2c2766e41e39ccba49697b2ecb7, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.165702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d150bd-fea6-44f5-b40f-43134bd88cdd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.242574 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 940/1845] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:22 2022] PUT /identity/v3/projects/cb67074d843a44ef857b0d706303e477/users/e0aff2c2766e41e39ccba49697b2ecb7/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:22.246712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293437be-13d7-411b-aeaa-806690d3f085 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.257334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293437be-13d7-411b-aeaa-806690d3f085 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.322132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d257d3a-7540-4f51-b090-cd4f2b935f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.322834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d257d3a-7540-4f51-b090-cd4f2b935f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.323011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d257d3a-7540-4f51-b090-cd4f2b935f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.323279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d257d3a-7540-4f51-b090-cd4f2b935f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.326320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d257d3a-7540-4f51-b090-cd4f2b935f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.328367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d257d3a-7540-4f51-b090-cd4f2b935f97 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.343408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 906/1846] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:22.348052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d150bd-fea6-44f5-b40f-43134bd88cdd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.355427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d150bd-fea6-44f5-b40f-43134bd88cdd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.360554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe35e-aeeb-4d6b-a852-27421ee6407f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.361203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe35e-aeeb-4d6b-a852-27421ee6407f admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.361517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe35e-aeeb-4d6b-a852-27421ee6407f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.361794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe35e-aeeb-4d6b-a852-27421ee6407f admin admin] PATH_INFO: `/v3/projects/cb67074d843a44ef857b0d706303e477/users/e0aff2c2766e41e39ccba49697b2ecb7/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.363704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fe35e-aeeb-4d6b-a852-27421ee6407f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb67074d843a44ef857b0d706303e477, user_id=e0aff2c2766e41e39ccba49697b2ecb7, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.366535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fe35e-aeeb-4d6b-a852-27421ee6407f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.376952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 941/1847] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:22 2022] PUT /identity/v3/projects/cb67074d843a44ef857b0d706303e477/users/e0aff2c2766e41e39ccba49697b2ecb7/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:22.391996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49d10-c7c1-4122-819d-ec7ee018053f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.392283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49d10-c7c1-4122-819d-ec7ee018053f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.392684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c49d10-c7c1-4122-819d-ec7ee018053f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.404626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:22.425008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-74c49d10-c7c1-4122-819d-ec7ee018053f None None] MFA Rules not processed for user `e0aff2c2766e41e39ccba49697b2ecb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:22.426262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-74c49d10-c7c1-4122-819d-ec7ee018053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:22.496752 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 907/1848] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:22.848605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe35e-aeeb-4d6b-a852-27421ee6407f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.914303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe35e-aeeb-4d6b-a852-27421ee6407f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.917938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fd575-f83d-470e-aefb-6d36f28dc0a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.918544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fd575-f83d-470e-aefb-6d36f28dc0a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.918820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fd575-f83d-470e-aefb-6d36f28dc0a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.919089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fd575-f83d-470e-aefb-6d36f28dc0a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.929010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fd575-f83d-470e-aefb-6d36f28dc0a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.930453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fd575-f83d-470e-aefb-6d36f28dc0a7 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.940550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 942/1849] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:23.086108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:23.589947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49d10-c7c1-4122-819d-ec7ee018053f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:23.651315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c49d10-c7c1-4122-819d-ec7ee018053f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:23.656753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:23.657387 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.657713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.658048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] PATH_INFO: `/v3/users/816e74d815c5466c8150a03b7150dce3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.659193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] RBAC: Authorizing `identity:delete_user(user_id=816e74d815c5466c8150a03b7150dce3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:23.661161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:23.794131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:23.858716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '816e74d815c5466c8150a03b7150dce3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:23.859363 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '816e74d815c5466c8150a03b7150dce3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:23.942826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '816e74d815c5466c8150a03b7150dce3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:23.956976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '816e74d815c5466c8150a03b7150dce3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:23.964214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '816e74d815c5466c8150a03b7150dce3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:23.971431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 908/1850] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:23 2022] DELETE /identity/v3/users/816e74d815c5466c8150a03b7150dce3 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:23.976063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23575e-01d5-4446-bfec-09deb39ada46 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.976558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23575e-01d5-4446-bfec-09deb39ada46 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.976651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23575e-01d5-4446-bfec-09deb39ada46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.996354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5f23575e-01d5-4446-bfec-09deb39ada46 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:23.997824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5f23575e-01d5-4446-bfec-09deb39ada46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:24.065015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 943/1851] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:24.072048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:24.073494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:24.140417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544293aa-77d3-46ff-91a2-f750bdb8107c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:24.143997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fdc17-1762-479c-aa48-9dcfe8abbcb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:24.144611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fdc17-1762-479c-aa48-9dcfe8abbcb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:24.144911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fdc17-1762-479c-aa48-9dcfe8abbcb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:24.145167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fdc17-1762-479c-aa48-9dcfe8abbcb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:24.155231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fdc17-1762-479c-aa48-9dcfe8abbcb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:24.156859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fdc17-1762-479c-aa48-9dcfe8abbcb5 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:24.171378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 909/1852] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:24.508310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:24.585647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23575e-01d5-4446-bfec-09deb39ada46 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:24.587080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5f23575e-01d5-4446-bfec-09deb39ada46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:24.599567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23575e-01d5-4446-bfec-09deb39ada46 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:24.604575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:24.605219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:24.605575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:24.605853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] PATH_INFO: `/v3/projects/d9a5e4b1c7d649a1bd9de83bf3e81a9d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:24.607138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a5e4b1c7d649a1bd9de83bf3e81a9d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:24.609070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:24.717294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a5e4b1c7d649a1bd9de83bf3e81a9d', 'request_id': 'req-5eb087c4-3891-4953-922f-bf945c18d8d7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:24.799949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a5e4b1c7d649a1bd9de83bf3e81a9d', 'request_id': 'req-5eb087c4-3891-4953-922f-bf945c18d8d7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:24.806770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a5e4b1c7d649a1bd9de83bf3e81a9d', 'request_id': 'req-5eb087c4-3891-4953-922f-bf945c18d8d7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:24.817559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] The token cache is being invalidate because project d9a5e4b1c7d649a1bd9de83bf3e81a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:24.817847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:24.818555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:24.820012 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 944/1853] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:24 2022] DELETE /identity/v3/projects/d9a5e4b1c7d649a1bd9de83bf3e81a9d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:24.824051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fdc17-1762-479c-aa48-9dcfe8abbcb5 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:24.834520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fdc17-1762-479c-aa48-9dcfe8abbcb5 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:24.896855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:24.897516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:24.897802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:24.898080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] PATH_INFO: `/v3/users/e0aff2c2766e41e39ccba49697b2ecb7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:24.899210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0aff2c2766e41e39ccba49697b2ecb7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:24.901290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:25.022628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:25.090659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0aff2c2766e41e39ccba49697b2ecb7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:25.091548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0aff2c2766e41e39ccba49697b2ecb7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:25.171556 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0aff2c2766e41e39ccba49697b2ecb7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:25.186698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0aff2c2766e41e39ccba49697b2ecb7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:25.194003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a18f1185-3a63-4470-827e-78accaff6fd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0aff2c2766e41e39ccba49697b2ecb7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:25.203773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 910/1854] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:24 2022] DELETE /identity/v3/users/e0aff2c2766e41e39ccba49697b2ecb7 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:25.347453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:25.644479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:25.658373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:25.659529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb087c4-3891-4953-922f-bf945c18d8d7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:25.721322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:25.722119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:25.722430 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:25.722709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] PATH_INFO: `/v3/projects/cb67074d843a44ef857b0d706303e477` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:25.724003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb67074d843a44ef857b0d706303e477)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:25.726150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:25.732621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:25.762720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:25.850552 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb67074d843a44ef857b0d706303e477', 'request_id': 'req-c569f393-1a28-4357-ada8-2c323b5cbae6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:25.928269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb67074d843a44ef857b0d706303e477', 'request_id': 'req-c569f393-1a28-4357-ada8-2c323b5cbae6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:25.935963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb67074d843a44ef857b0d706303e477', 'request_id': 'req-c569f393-1a28-4357-ada8-2c323b5cbae6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:25.946649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] The token cache is being invalidate because project cb67074d843a44ef857b0d706303e477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:25.946938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:25.947593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c569f393-1a28-4357-ada8-2c323b5cbae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:25.949089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 945/1855] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:25 2022] DELETE /identity/v3/projects/cb67074d843a44ef857b0d706303e477 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:25.954093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b50d0-d90f-48b8-ac5f-d305a2ed5cba None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:25.957603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b50d0-d90f-48b8-ac5f-d305a2ed5cba None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:25.957603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b50d0-d90f-48b8-ac5f-d305a2ed5cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:25.975772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-745b50d0-d90f-48b8-ac5f-d305a2ed5cba None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:25.977195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-745b50d0-d90f-48b8-ac5f-d305a2ed5cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:26.029344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:26.039960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 911/1856] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:26.050093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be2bef-728c-4c1b-bd60-402fd963d6b5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.050402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be2bef-728c-4c1b-bd60-402fd963d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.050682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be2bef-728c-4c1b-bd60-402fd963d6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.071297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-63be2bef-728c-4c1b-bd60-402fd963d6b5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:26.072505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-63be2bef-728c-4c1b-bd60-402fd963d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:26.081147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 946/1857] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:26.085371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b50d0-d90f-48b8-ac5f-d305a2ed5cba None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:26.095966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b50d0-d90f-48b8-ac5f-d305a2ed5cba None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:26.106977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c1c9b-bdd4-4fc0-ad42-af65d88b5dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:26.106977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c1c9b-bdd4-4fc0-ad42-af65d88b5dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.106977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c1c9b-bdd4-4fc0-ad42-af65d88b5dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.106977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c1c9b-bdd4-4fc0-ad42-af65d88b5dd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.107365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c1c9b-bdd4-4fc0-ad42-af65d88b5dd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:26.107365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c1c9b-bdd4-4fc0-ad42-af65d88b5dd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:26.107365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c1c9b-bdd4-4fc0-ad42-af65d88b5dd9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:26.116255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 912/1858] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:31:26.120519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be2bef-728c-4c1b-bd60-402fd963d6b5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:26.130167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be2bef-728c-4c1b-bd60-402fd963d6b5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:26.134914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc6ab7-deaa-4947-8cab-aa53b0edb105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:26.135337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc6ab7-deaa-4947-8cab-aa53b0edb105 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.135615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc6ab7-deaa-4947-8cab-aa53b0edb105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.135887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc6ab7-deaa-4947-8cab-aa53b0edb105 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.136514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc6ab7-deaa-4947-8cab-aa53b0edb105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:26.139278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc6ab7-deaa-4947-8cab-aa53b0edb105 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:26.182542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:26.218562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 947/1859] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:26 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:26.222800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c1c9b-bdd4-4fc0-ad42-af65d88b5dd9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:26.233394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c1c9b-bdd4-4fc0-ad42-af65d88b5dd9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:26.293553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48048f38-81e0-456d-afd3-12b541db5864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:26.294692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48048f38-81e0-456d-afd3-12b541db5864 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.294969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48048f38-81e0-456d-afd3-12b541db5864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.295265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48048f38-81e0-456d-afd3-12b541db5864 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.295844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48048f38-81e0-456d-afd3-12b541db5864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:26.297798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48048f38-81e0-456d-afd3-12b541db5864 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:26.391499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 913/1860] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:26 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:26.395622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc6ab7-deaa-4947-8cab-aa53b0edb105 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:26.406507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc6ab7-deaa-4947-8cab-aa53b0edb105 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:26.409009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:26.411239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cface-0c17-4f5c-bc7e-4b2ed04f47b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:26.411834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cface-0c17-4f5c-bc7e-4b2ed04f47b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.412109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cface-0c17-4f5c-bc7e-4b2ed04f47b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.412378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cface-0c17-4f5c-bc7e-4b2ed04f47b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.413015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cface-0c17-4f5c-bc7e-4b2ed04f47b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:26.414665 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cface-0c17-4f5c-bc7e-4b2ed04f47b3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:26.425198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 948/1861] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:26.429357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48048f38-81e0-456d-afd3-12b541db5864 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:26.440264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48048f38-81e0-456d-afd3-12b541db5864 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:26.445721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff06780-ed92-48f4-a145-30d52473f516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:26.445721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff06780-ed92-48f4-a145-30d52473f516 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.445721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff06780-ed92-48f4-a145-30d52473f516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.446032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff06780-ed92-48f4-a145-30d52473f516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.446620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff06780-ed92-48f4-a145-30d52473f516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:26.448094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff06780-ed92-48f4-a145-30d52473f516 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:26.460075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 914/1862] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:26.464794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cface-0c17-4f5c-bc7e-4b2ed04f47b3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:26.480076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cface-0c17-4f5c-bc7e-4b2ed04f47b3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:26.483183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e3a11-1b61-46d8-baf5-a544ec507a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:26.483884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3a11-1b61-46d8-baf5-a544ec507a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.484166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3a11-1b61-46d8-baf5-a544ec507a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.484443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3a11-1b61-46d8-baf5-a544ec507a49 admin admin] PATH_INFO: `/v3/projects/c5bba353564742ec830a40b2e7f4d2ee/users/d62d15658cd249738730202990f42ddf/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.508410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e3a11-1b61-46d8-baf5-a544ec507a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bba353564742ec830a40b2e7f4d2ee, user_id=d62d15658cd249738730202990f42ddf, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:26.511237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e3a11-1b61-46d8-baf5-a544ec507a49 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:26.529141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:26.597526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 949/1863] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:26 2022] PUT /identity/v3/projects/c5bba353564742ec830a40b2e7f4d2ee/users/d62d15658cd249738730202990f42ddf/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:26.604856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:26.608165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fbd2a-aaec-44c6-aa7b-d5b39db37eb5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.608442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fbd2a-aaec-44c6-aa7b-d5b39db37eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.608713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fbd2a-aaec-44c6-aa7b-d5b39db37eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.637487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-876fbd2a-aaec-44c6-aa7b-d5b39db37eb5 None None] MFA Rules not processed for user `d62d15658cd249738730202990f42ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:26.639301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-876fbd2a-aaec-44c6-aa7b-d5b39db37eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:26.706640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 915/1864] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:26.721216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e3a11-1b61-46d8-baf5-a544ec507a49 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:26.725344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5f8e3a11-1b61-46d8-baf5-a544ec507a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:26.754405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:26.781700 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:26.788819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e3a11-1b61-46d8-baf5-a544ec507a49 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:26.792719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6180e-5964-421f-a4e4-49c95c2cf374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:26.793336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6180e-5964-421f-a4e4-49c95c2cf374 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.793652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6180e-5964-421f-a4e4-49c95c2cf374 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.793936 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6180e-5964-421f-a4e4-49c95c2cf374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.804497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6180e-5964-421f-a4e4-49c95c2cf374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:26.805991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6180e-5964-421f-a4e4-49c95c2cf374 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:26.816924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 950/1865] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:26.838586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fbd2a-aaec-44c6-aa7b-d5b39db37eb5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:26.904203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fbd2a-aaec-44c6-aa7b-d5b39db37eb5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:26.909861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:26.910494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.910769 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.911039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] PATH_INFO: `/v3/users/d62d15658cd249738730202990f42ddf` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.912166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62d15658cd249738730202990f42ddf)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:26.914192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:26.935268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:27.131011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:27.144968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62d15658cd249738730202990f42ddf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:27.145666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62d15658cd249738730202990f42ddf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:27.213985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62d15658cd249738730202990f42ddf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:27.227301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62d15658cd249738730202990f42ddf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:27.234987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62d15658cd249738730202990f42ddf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:27.244219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 916/1866] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:26 2022] DELETE /identity/v3/users/d62d15658cd249738730202990f42ddf => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:27.247767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce0fa-45a3-4c73-93fe-7c14f7d30b6a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:27.247767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce0fa-45a3-4c73-93fe-7c14f7d30b6a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:27.247767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce0fa-45a3-4c73-93fe-7c14f7d30b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:27.268490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-510ce0fa-45a3-4c73-93fe-7c14f7d30b6a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:27.273442 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-510ce0fa-45a3-4c73-93fe-7c14f7d30b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:27.342048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 951/1867] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:27.366139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:27.367599 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:27.400791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:27.427051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0896d3-0197-47d2-94f7-62e5901d22b2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:27.431150 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d69bb-8d38-4752-b887-120a05ee26c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:27.431751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d69bb-8d38-4752-b887-120a05ee26c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:27.432075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d69bb-8d38-4752-b887-120a05ee26c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:27.432343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d69bb-8d38-4752-b887-120a05ee26c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:27.442267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d69bb-8d38-4752-b887-120a05ee26c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:27.443751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d69bb-8d38-4752-b887-120a05ee26c4 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:27.454143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 917/1868] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:27.843748 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ce0fa-45a3-4c73-93fe-7c14f7d30b6a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:27.845306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-510ce0fa-45a3-4c73-93fe-7c14f7d30b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:27.858985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ce0fa-45a3-4c73-93fe-7c14f7d30b6a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:27.864002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:27.864672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:27.864956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:27.865227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] PATH_INFO: `/v3/projects/c5bba353564742ec830a40b2e7f4d2ee` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:27.866859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5bba353564742ec830a40b2e7f4d2ee)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:27.868780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:27.968657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5bba353564742ec830a40b2e7f4d2ee', 'request_id': 'req-ff92c278-d7aa-47ad-8675-d7739c69cb82'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:27.970347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:28.077211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5bba353564742ec830a40b2e7f4d2ee', 'request_id': 'req-ff92c278-d7aa-47ad-8675-d7739c69cb82'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:28.086565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5bba353564742ec830a40b2e7f4d2ee', 'request_id': 'req-ff92c278-d7aa-47ad-8675-d7739c69cb82'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:28.108268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] The token cache is being invalidate because project c5bba353564742ec830a40b2e7f4d2ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:28.108815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:28.109648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff92c278-d7aa-47ad-8675-d7739c69cb82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:28.111824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 952/1869] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:27 2022] DELETE /identity/v3/projects/c5bba353564742ec830a40b2e7f4d2ee => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:28.116478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af356e8-4977-498d-a066-785be2b97422 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:28.117026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af356e8-4977-498d-a066-785be2b97422 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:28.117298 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af356e8-4977-498d-a066-785be2b97422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:28.139232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1af356e8-4977-498d-a066-785be2b97422 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:28.141575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1af356e8-4977-498d-a066-785be2b97422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:28.209026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 918/1870] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:28.216693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7026-ac99-45eb-83cc-ee87cfa7cde0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:28.217081 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7026-ac99-45eb-83cc-ee87cfa7cde0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:28.217423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7026-ac99-45eb-83cc-ee87cfa7cde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:28.236962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-339f7026-ac99-45eb-83cc-ee87cfa7cde0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:28.238291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-339f7026-ac99-45eb-83cc-ee87cfa7cde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:28.246824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 953/1871] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:28.251567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af356e8-4977-498d-a066-785be2b97422 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:28.263434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af356e8-4977-498d-a066-785be2b97422 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:28.269070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9cc17-6df4-45d2-bc0f-2f11aa7d8c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:28.269786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9cc17-6df4-45d2-bc0f-2f11aa7d8c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:28.270063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9cc17-6df4-45d2-bc0f-2f11aa7d8c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:28.270594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9cc17-6df4-45d2-bc0f-2f11aa7d8c66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:28.271148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9cc17-6df4-45d2-bc0f-2f11aa7d8c66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:28.272919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9cc17-6df4-45d2-bc0f-2f11aa7d8c66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:28.274613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9cc17-6df4-45d2-bc0f-2f11aa7d8c66 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:28.289514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 919/1872] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:31:28.291883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f7026-ac99-45eb-83cc-ee87cfa7cde0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:28.302103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f7026-ac99-45eb-83cc-ee87cfa7cde0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:28.306650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b214ac-c825-41a0-bc00-07803b13be97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:28.307247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b214ac-c825-41a0-bc00-07803b13be97 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:28.307521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b214ac-c825-41a0-bc00-07803b13be97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:28.307789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b214ac-c825-41a0-bc00-07803b13be97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:28.308420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b214ac-c825-41a0-bc00-07803b13be97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:28.311795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b214ac-c825-41a0-bc00-07803b13be97 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:28.400569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 954/1873] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:28 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:28.404935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9cc17-6df4-45d2-bc0f-2f11aa7d8c66 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:28.418064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9cc17-6df4-45d2-bc0f-2f11aa7d8c66 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:28.495778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2b3aa-eada-443b-b93e-45c7400fa0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:28.498059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2b3aa-eada-443b-b93e-45c7400fa0bc admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:28.498566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2b3aa-eada-443b-b93e-45c7400fa0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:28.499048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2b3aa-eada-443b-b93e-45c7400fa0bc admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:28.500270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2b3aa-eada-443b-b93e-45c7400fa0bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:28.503685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2b3aa-eada-443b-b93e-45c7400fa0bc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:28.635132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 920/1874] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:28 2022] POST /identity/v3/users => generated 288 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:28.636374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b214ac-c825-41a0-bc00-07803b13be97 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:28.647131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b214ac-c825-41a0-bc00-07803b13be97 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:28.652027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed6a97-5311-44aa-ba90-283dcba6b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:28.652634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed6a97-5311-44aa-ba90-283dcba6b6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:28.652917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed6a97-5311-44aa-ba90-283dcba6b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:28.653915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed6a97-5311-44aa-ba90-283dcba6b6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:28.654817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed6a97-5311-44aa-ba90-283dcba6b6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:28.656547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed6a97-5311-44aa-ba90-283dcba6b6a9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:28.669515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 955/1875] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:28.671333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2b3aa-eada-443b-b93e-45c7400fa0bc admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:28.684080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2b3aa-eada-443b-b93e-45c7400fa0bc admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:28.687518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c5fe4-28d5-4626-80b5-d6919d09e23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:28.688126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c5fe4-28d5-4626-80b5-d6919d09e23f admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:28.688402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c5fe4-28d5-4626-80b5-d6919d09e23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:28.688675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c5fe4-28d5-4626-80b5-d6919d09e23f admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:28.689314 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c5fe4-28d5-4626-80b5-d6919d09e23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:28.690977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c5fe4-28d5-4626-80b5-d6919d09e23f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:28.702625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 921/1876] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:28.707116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed6a97-5311-44aa-ba90-283dcba6b6a9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:28.717315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed6a97-5311-44aa-ba90-283dcba6b6a9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:28.721901 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afafd3b-c5b6-4958-9e8b-cf9ea0000b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:28.722557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afafd3b-c5b6-4958-9e8b-cf9ea0000b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:28.722865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afafd3b-c5b6-4958-9e8b-cf9ea0000b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:28.723145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afafd3b-c5b6-4958-9e8b-cf9ea0000b19 admin admin] PATH_INFO: `/v3/projects/b4c04fe6be9347489ede9d278bfb3570/users/af0eb6dc908c4a4382bf8371da0e5e18/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:28.738511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afafd3b-c5b6-4958-9e8b-cf9ea0000b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c04fe6be9347489ede9d278bfb3570, user_id=af0eb6dc908c4a4382bf8371da0e5e18, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:28.741406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afafd3b-c5b6-4958-9e8b-cf9ea0000b19 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:28.878320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 956/1877] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:28 2022] PUT /identity/v3/projects/b4c04fe6be9347489ede9d278bfb3570/users/af0eb6dc908c4a4382bf8371da0e5e18/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:28.889946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9efc0a-4e3a-4f94-8a34-3c5de0ff6304 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:28.890681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9efc0a-4e3a-4f94-8a34-3c5de0ff6304 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:28.890681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9efc0a-4e3a-4f94-8a34-3c5de0ff6304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:28.922221 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9d9efc0a-4e3a-4f94-8a34-3c5de0ff6304 None None] MFA Rules not processed for user `af0eb6dc908c4a4382bf8371da0e5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:28.922402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9d9efc0a-4e3a-4f94-8a34-3c5de0ff6304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:28.956556 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:28.999097 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 922/1878] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:29.010138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afafd3b-c5b6-4958-9e8b-cf9ea0000b19 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:29.011810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6afafd3b-c5b6-4958-9e8b-cf9ea0000b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:29.069904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afafd3b-c5b6-4958-9e8b-cf9ea0000b19 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:29.073639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f288f4-b2a7-4f50-9d5c-e4ed2c8683ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:29.078051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f288f4-b2a7-4f50-9d5c-e4ed2c8683ac service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:29.078051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f288f4-b2a7-4f50-9d5c-e4ed2c8683ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:29.078051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f288f4-b2a7-4f50-9d5c-e4ed2c8683ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:29.083238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f288f4-b2a7-4f50-9d5c-e4ed2c8683ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:29.084658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f288f4-b2a7-4f50-9d5c-e4ed2c8683ac service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:29.093761 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 957/1879] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:39.073443 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=16 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:39.089417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=59 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:46.320406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9efc0a-4e3a-4f94-8a34-3c5de0ff6304 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:46.388357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9efc0a-4e3a-4f94-8a34-3c5de0ff6304 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:46.393826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:46.394461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.394736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.395054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] PATH_INFO: `/v3/users/af0eb6dc908c4a4382bf8371da0e5e18` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.396149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0eb6dc908c4a4382bf8371da0e5e18)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:46.398122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:46.623916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0eb6dc908c4a4382bf8371da0e5e18'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:46.624657 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0eb6dc908c4a4382bf8371da0e5e18'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:46.684293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0eb6dc908c4a4382bf8371da0e5e18'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:46.697762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0eb6dc908c4a4382bf8371da0e5e18'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:46.705893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0eb6dc908c4a4382bf8371da0e5e18'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:46.713230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 923/1880] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:46 2022] DELETE /identity/v3/users/af0eb6dc908c4a4382bf8371da0e5e18 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:46.720485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d4018-fd12-4e9b-9e6d-9ea30a236bce None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.721040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d4018-fd12-4e9b-9e6d-9ea30a236bce None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.721040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d4018-fd12-4e9b-9e6d-9ea30a236bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.741428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-111d4018-fd12-4e9b-9e6d-9ea30a236bce None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:46.742608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-111d4018-fd12-4e9b-9e6d-9ea30a236bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:46.800409 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 958/1881] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:46.811755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:46.813712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:46.874583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a899fa1-4103-4aee-9a15-fab5cdd411f9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:46.878282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d253314-238e-4965-aea1-01edae2c34e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:46.879024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d253314-238e-4965-aea1-01edae2c34e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.879329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d253314-238e-4965-aea1-01edae2c34e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.879583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d253314-238e-4965-aea1-01edae2c34e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.890332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d253314-238e-4965-aea1-01edae2c34e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:46.891786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d253314-238e-4965-aea1-01edae2c34e4 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:46.901402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 924/1882] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:47.385829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d4018-fd12-4e9b-9e6d-9ea30a236bce None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.386717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-111d4018-fd12-4e9b-9e6d-9ea30a236bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:47.402038 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d4018-fd12-4e9b-9e6d-9ea30a236bce None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.408847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.409545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.409827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.410102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] PATH_INFO: `/v3/projects/b4c04fe6be9347489ede9d278bfb3570` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.412647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c04fe6be9347489ede9d278bfb3570)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.413674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.454488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38e511-314c-46ac-95f7-36a616d646ff None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.455290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38e511-314c-46ac-95f7-36a616d646ff None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.455290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38e511-314c-46ac-95f7-36a616d646ff None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.457082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 925/1883] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:31:47.479916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e2c86-6dae-46ff-850e-c344f03cd26f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.480228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e2c86-6dae-46ff-850e-c344f03cd26f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.480570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e2c86-6dae-46ff-850e-c344f03cd26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.508559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0b6e2c86-6dae-46ff-850e-c344f03cd26f None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:47.509810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0b6e2c86-6dae-46ff-850e-c344f03cd26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:47.531126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:47.573665 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c04fe6be9347489ede9d278bfb3570', 'request_id': 'req-52d6b489-116f-4958-ab14-b3e96edc5bf0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:47.595372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 926/1884] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:31:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:47.596311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e2c86-6dae-46ff-850e-c344f03cd26f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.597619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0b6e2c86-6dae-46ff-850e-c344f03cd26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:47.612352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e2c86-6dae-46ff-850e-c344f03cd26f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.644194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c04fe6be9347489ede9d278bfb3570', 'request_id': 'req-52d6b489-116f-4958-ab14-b3e96edc5bf0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:47.651637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c04fe6be9347489ede9d278bfb3570', 'request_id': 'req-52d6b489-116f-4958-ab14-b3e96edc5bf0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:47.664804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] The token cache is being invalidate because project b4c04fe6be9347489ede9d278bfb3570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:47.665177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:47.666180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:47.667787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 959/1885] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:47 2022] DELETE /identity/v3/projects/b4c04fe6be9347489ede9d278bfb3570 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:47.671873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.671873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:47.676456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.677786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.678077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.678347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] PATH_INFO: `/v3/users/e6e8fc72d34e430b92930e3a40028861` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.679526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e8fc72d34e430b92930e3a40028861)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.681767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.749084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6b489-116f-4958-ab14-b3e96edc5bf0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.754795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056a4f3-2e17-4366-a780-d12e44dec2d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.755436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056a4f3-2e17-4366-a780-d12e44dec2d7 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.755764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056a4f3-2e17-4366-a780-d12e44dec2d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.756045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056a4f3-2e17-4366-a780-d12e44dec2d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.757809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a056a4f3-2e17-4366-a780-d12e44dec2d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:47.770695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056a4f3-2e17-4366-a780-d12e44dec2d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.772286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056a4f3-2e17-4366-a780-d12e44dec2d7 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.783235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 960/1886] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:47.784633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694d664-7313-49e6-a140-97df81818bf2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.784929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694d664-7313-49e6-a140-97df81818bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.785371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694d664-7313-49e6-a140-97df81818bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.815280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1694d664-7313-49e6-a140-97df81818bf2 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:47.816657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1694d664-7313-49e6-a140-97df81818bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:47.826543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 961/1887] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:47.833934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e9a2-c141-4998-9d7b-5ebb1ad74f1a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.834220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e9a2-c141-4998-9d7b-5ebb1ad74f1a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.834499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e9a2-c141-4998-9d7b-5ebb1ad74f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.842483 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:47.855322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9614e9a2-c141-4998-9d7b-5ebb1ad74f1a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:47.857076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9614e9a2-c141-4998-9d7b-5ebb1ad74f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:47.865151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 962/1888] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:47.869607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614e9a2-c141-4998-9d7b-5ebb1ad74f1a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.879902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614e9a2-c141-4998-9d7b-5ebb1ad74f1a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.884654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4df45d-d9d2-4438-b169-0bb1ab983538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.885256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4df45d-d9d2-4438-b169-0bb1ab983538 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.885569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4df45d-d9d2-4438-b169-0bb1ab983538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.885842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4df45d-d9d2-4438-b169-0bb1ab983538 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.885947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e8fc72d34e430b92930e3a40028861'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:47.886485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4df45d-d9d2-4438-b169-0bb1ab983538 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:47.886721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e8fc72d34e430b92930e3a40028861'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:47.886905 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4df45d-d9d2-4438-b169-0bb1ab983538 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.888705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4df45d-d9d2-4438-b169-0bb1ab983538 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.905587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 963/1889] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:31:47.907388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4df45d-d9d2-4438-b169-0bb1ab983538 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.909112 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ee4df45d-d9d2-4438-b169-0bb1ab983538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:47.923100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4df45d-d9d2-4438-b169-0bb1ab983538 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.928860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13376fc-4121-4cff-af02-ebfa3c82de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.929490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13376fc-4121-4cff-af02-ebfa3c82de14 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.933199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13376fc-4121-4cff-af02-ebfa3c82de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.933199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13376fc-4121-4cff-af02-ebfa3c82de14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.933199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13376fc-4121-4cff-af02-ebfa3c82de14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.933199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13376fc-4121-4cff-af02-ebfa3c82de14 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.952907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e8fc72d34e430b92930e3a40028861'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:47.968074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e8fc72d34e430b92930e3a40028861'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:47.976026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-be2d5c00-0d6f-493e-9bce-c680ae30cd72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e8fc72d34e430b92930e3a40028861'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:47.983991 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 927/1890] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:47 2022] DELETE /identity/v3/users/e6e8fc72d34e430b92930e3a40028861 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:47.988226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3967f-a272-454b-8477-691788b84c18 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.988536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3967f-a272-454b-8477-691788b84c18 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.988811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3967f-a272-454b-8477-691788b84c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:48.008075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6ed3967f-a272-454b-8477-691788b84c18 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:48.009279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6ed3967f-a272-454b-8477-691788b84c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:48.039430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:48.046471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 964/1891] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:47 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:48.050959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13376fc-4121-4cff-af02-ebfa3c82de14 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:48.081822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 928/1892] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:48.094215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3967f-a272-454b-8477-691788b84c18 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:48.096046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6ed3967f-a272-454b-8477-691788b84c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:48.139434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13376fc-4121-4cff-af02-ebfa3c82de14 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:48.144455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cd439-26fa-4a02-80f7-897efac9c5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:48.145100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd439-26fa-4a02-80f7-897efac9c5ef admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:48.145480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd439-26fa-4a02-80f7-897efac9c5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:48.145923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd439-26fa-4a02-80f7-897efac9c5ef admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:48.146709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cd439-26fa-4a02-80f7-897efac9c5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:48.148793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cd439-26fa-4a02-80f7-897efac9c5ef admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:48.166943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3967f-a272-454b-8477-691788b84c18 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:48.228669 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61d569-58ef-4c01-b02a-59d30e43c306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:48.229284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61d569-58ef-4c01-b02a-59d30e43c306 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:48.229720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61d569-58ef-4c01-b02a-59d30e43c306 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:48.232347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61d569-58ef-4c01-b02a-59d30e43c306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:48.240513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61d569-58ef-4c01-b02a-59d30e43c306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:48.242059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61d569-58ef-4c01-b02a-59d30e43c306 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:48.257259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 929/1893] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:48.260803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 965/1894] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:48 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:48.261078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61d569-58ef-4c01-b02a-59d30e43c306 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:48.272246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61d569-58ef-4c01-b02a-59d30e43c306 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:48.294838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dd974-c01b-4925-b379-d0b17e8b5dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:48.294838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dd974-c01b-4925-b379-d0b17e8b5dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:48.294838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dd974-c01b-4925-b379-d0b17e8b5dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:48.294838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dd974-c01b-4925-b379-d0b17e8b5dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:48.295155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dd974-c01b-4925-b379-d0b17e8b5dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:48.295155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dd974-c01b-4925-b379-d0b17e8b5dc3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:48.311764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 930/1895] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:48.315858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cd439-26fa-4a02-80f7-897efac9c5ef admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:48.329291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cd439-26fa-4a02-80f7-897efac9c5ef admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:48.337236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0eb7d-eac8-4050-ab21-acde4247d9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:48.338029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0eb7d-eac8-4050-ab21-acde4247d9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:48.338512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0eb7d-eac8-4050-ab21-acde4247d9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:48.338853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0eb7d-eac8-4050-ab21-acde4247d9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:48.340680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0eb7d-eac8-4050-ab21-acde4247d9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:48.343376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0eb7d-eac8-4050-ab21-acde4247d9d3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:48.360122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 966/1896] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:48.364927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dd974-c01b-4925-b379-d0b17e8b5dc3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:48.376720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dd974-c01b-4925-b379-d0b17e8b5dc3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:48.382162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8026d88-fe97-44fc-a2a3-1eb59b6c4fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:48.382938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8026d88-fe97-44fc-a2a3-1eb59b6c4fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:48.383232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8026d88-fe97-44fc-a2a3-1eb59b6c4fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:48.383659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8026d88-fe97-44fc-a2a3-1eb59b6c4fd7 admin admin] PATH_INFO: `/v3/projects/289727701c414a53917eb1de18fe1c5a/users/5fa12d4b835d4b59924f1ebec27f251b/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:48.402557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8026d88-fe97-44fc-a2a3-1eb59b6c4fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289727701c414a53917eb1de18fe1c5a, user_id=5fa12d4b835d4b59924f1ebec27f251b, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:48.405306 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8026d88-fe97-44fc-a2a3-1eb59b6c4fd7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:48.463956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 931/1897] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:48 2022] PUT /identity/v3/projects/289727701c414a53917eb1de18fe1c5a/users/5fa12d4b835d4b59924f1ebec27f251b/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:48.477923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e9b67-da4b-4024-8b18-9b5907749632 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:48.478177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e9b67-da4b-4024-8b18-9b5907749632 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:48.478503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e9b67-da4b-4024-8b18-9b5907749632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:48.529937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-060e9b67-da4b-4024-8b18-9b5907749632 None None] MFA Rules not processed for user `5fa12d4b835d4b59924f1ebec27f251b`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:48.531428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-060e9b67-da4b-4024-8b18-9b5907749632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:48.607864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 967/1898] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:48.608703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8026d88-fe97-44fc-a2a3-1eb59b6c4fd7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:48.688862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:48.693493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8026d88-fe97-44fc-a2a3-1eb59b6c4fd7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:48.698264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034b66d-b757-4ae1-8523-981f1f345fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:48.698872 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034b66d-b757-4ae1-8523-981f1f345fdd admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:48.699682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034b66d-b757-4ae1-8523-981f1f345fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:48.700017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034b66d-b757-4ae1-8523-981f1f345fdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:48.700658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034b66d-b757-4ae1-8523-981f1f345fdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:48.702605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034b66d-b757-4ae1-8523-981f1f345fdd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:48.781522 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 932/1899] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:48 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:48.783612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e9b67-da4b-4024-8b18-9b5907749632 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:48.796167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e9b67-da4b-4024-8b18-9b5907749632 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:48.831764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:48.885826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81429f-61fb-4958-80af-0232a0ad2bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:48.887681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81429f-61fb-4958-80af-0232a0ad2bdd admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:48.888147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81429f-61fb-4958-80af-0232a0ad2bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:48.888573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81429f-61fb-4958-80af-0232a0ad2bdd admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:48.889395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81429f-61fb-4958-80af-0232a0ad2bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:48.893853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81429f-61fb-4958-80af-0232a0ad2bdd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.021433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 968/1900] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:48 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:49.026308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034b66d-b757-4ae1-8523-981f1f345fdd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.039182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034b66d-b757-4ae1-8523-981f1f345fdd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.044591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d7504-89cf-44c4-ab61-c6e4260cfca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.046003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d7504-89cf-44c4-ab61-c6e4260cfca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.046003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d7504-89cf-44c4-ab61-c6e4260cfca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.046003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d7504-89cf-44c4-ab61-c6e4260cfca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.046690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d7504-89cf-44c4-ab61-c6e4260cfca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.048294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d7504-89cf-44c4-ab61-c6e4260cfca9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.059041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 933/1901] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:49.063268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81429f-61fb-4958-80af-0232a0ad2bdd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.073881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81429f-61fb-4958-80af-0232a0ad2bdd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.079249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4034be-ba25-4f03-a3a5-1db474e25716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.079851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4034be-ba25-4f03-a3a5-1db474e25716 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.080139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4034be-ba25-4f03-a3a5-1db474e25716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.080440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4034be-ba25-4f03-a3a5-1db474e25716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.081081 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4034be-ba25-4f03-a3a5-1db474e25716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.082734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4034be-ba25-4f03-a3a5-1db474e25716 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.092857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 969/1902] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:49.104091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d7504-89cf-44c4-ab61-c6e4260cfca9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.107042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4034be-ba25-4f03-a3a5-1db474e25716 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.109167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-be4034be-ba25-4f03-a3a5-1db474e25716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:49.129425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4034be-ba25-4f03-a3a5-1db474e25716 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.132302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d7504-89cf-44c4-ab61-c6e4260cfca9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.136694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42ecbc-7efa-47c2-a50d-4bb41b8a57a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.137773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.138112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42ecbc-7efa-47c2-a50d-4bb41b8a57a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.138389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42ecbc-7efa-47c2-a50d-4bb41b8a57a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.138656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42ecbc-7efa-47c2-a50d-4bb41b8a57a7 admin admin] PATH_INFO: `/v3/projects/3324620b79f0449d86e5b5fbe666f028/users/2bf635af33494c458ba399a3e354a951/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.138962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.139340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.139738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] PATH_INFO: `/v3/projects/801ea44868d1482682628db4ae80f3ed` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.141004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=801ea44868d1482682628db4ae80f3ed)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.143271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.155979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42ecbc-7efa-47c2-a50d-4bb41b8a57a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3324620b79f0449d86e5b5fbe666f028, user_id=2bf635af33494c458ba399a3e354a951, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.159110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42ecbc-7efa-47c2-a50d-4bb41b8a57a7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.243705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 934/1903] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:49 2022] PUT /identity/v3/projects/3324620b79f0449d86e5b5fbe666f028/users/2bf635af33494c458ba399a3e354a951/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:49.255449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94311e21-b9d5-4fe2-8a7f-63c8f2ee3830 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.255947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94311e21-b9d5-4fe2-8a7f-63c8f2ee3830 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.256279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94311e21-b9d5-4fe2-8a7f-63c8f2ee3830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.260721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801ea44868d1482682628db4ae80f3ed', 'request_id': 'req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:49.293786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-94311e21-b9d5-4fe2-8a7f-63c8f2ee3830 None None] MFA Rules not processed for user `2bf635af33494c458ba399a3e354a951`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:49.293786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-94311e21-b9d5-4fe2-8a7f-63c8f2ee3830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:49.373403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 935/1904] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:49.374347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94311e21-b9d5-4fe2-8a7f-63c8f2ee3830 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.386916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801ea44868d1482682628db4ae80f3ed', 'request_id': 'req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:49.394842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801ea44868d1482682628db4ae80f3ed', 'request_id': 'req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:49.407045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] The token cache is being invalidate because project 801ea44868d1482682628db4ae80f3ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:49.407339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:49.408009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2c7bb89d-b1a6-427f-ad66-6dc46073c7f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:49.409710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 970/1905] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:49 2022] DELETE /identity/v3/projects/801ea44868d1482682628db4ae80f3ed => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:49.416053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362ef69-e7c3-4c74-abc7-143eebd4a9cf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.416348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362ef69-e7c3-4c74-abc7-143eebd4a9cf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.416617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362ef69-e7c3-4c74-abc7-143eebd4a9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.439948 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f362ef69-e7c3-4c74-abc7-143eebd4a9cf None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:49.441124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94311e21-b9d5-4fe2-8a7f-63c8f2ee3830 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.441240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f362ef69-e7c3-4c74-abc7-143eebd4a9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:49.445979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8ac23-07c6-4ce2-b418-10ab63a90cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.446552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8ac23-07c6-4ce2-b418-10ab63a90cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.447993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8ac23-07c6-4ce2-b418-10ab63a90cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.447993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8ac23-07c6-4ce2-b418-10ab63a90cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.447993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8ac23-07c6-4ce2-b418-10ab63a90cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.450274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 971/1906] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:49.450731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8ac23-07c6-4ce2-b418-10ab63a90cb5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.458053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c49676-2949-4358-b476-12903b17eb48 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.458339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c49676-2949-4358-b476-12903b17eb48 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.458617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c49676-2949-4358-b476-12903b17eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.483963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-46c49676-2949-4358-b476-12903b17eb48 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:49.485180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-46c49676-2949-4358-b476-12903b17eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:49.493197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 972/1907] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:49.501246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c49676-2949-4358-b476-12903b17eb48 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.509403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c49676-2949-4358-b476-12903b17eb48 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.515886 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d28dd-7df2-4354-85b4-6cb9d4ad07e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.516708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d28dd-7df2-4354-85b4-6cb9d4ad07e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.517040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d28dd-7df2-4354-85b4-6cb9d4ad07e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.517321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d28dd-7df2-4354-85b4-6cb9d4ad07e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.518238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d28dd-7df2-4354-85b4-6cb9d4ad07e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:49.519904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d28dd-7df2-4354-85b4-6cb9d4ad07e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.521944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d28dd-7df2-4354-85b4-6cb9d4ad07e6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.541390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 973/1908] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:31:49.545786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d28dd-7df2-4354-85b4-6cb9d4ad07e6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.556098 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d28dd-7df2-4354-85b4-6cb9d4ad07e6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.562774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac21d4b-1360-42d3-a341-7d407d0606c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.568056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac21d4b-1360-42d3-a341-7d407d0606c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.568056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac21d4b-1360-42d3-a341-7d407d0606c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.568056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac21d4b-1360-42d3-a341-7d407d0606c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.568056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac21d4b-1360-42d3-a341-7d407d0606c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.568056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac21d4b-1360-42d3-a341-7d407d0606c8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.597489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 936/1909] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:49 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:49.598978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8ac23-07c6-4ce2-b418-10ab63a90cb5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.600672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a6d8ac23-07c6-4ce2-b418-10ab63a90cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:49.626782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:49.669285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8ac23-07c6-4ce2-b418-10ab63a90cb5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.672687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 974/1910] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:49 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:49.676427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac21d4b-1360-42d3-a341-7d407d0606c8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.682717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c8b8b-b7f4-4bbf-a8f3-5e3ca22c80ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.683400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c8b8b-b7f4-4bbf-a8f3-5e3ca22c80ec admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.684984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c8b8b-b7f4-4bbf-a8f3-5e3ca22c80ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.685193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c8b8b-b7f4-4bbf-a8f3-5e3ca22c80ec admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.686077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c8b8b-b7f4-4bbf-a8f3-5e3ca22c80ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.689521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c8b8b-b7f4-4bbf-a8f3-5e3ca22c80ec admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.750681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:49.778565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 937/1911] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:49 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:49.783394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c8b8b-b7f4-4bbf-a8f3-5e3ca22c80ec admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.789010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac21d4b-1360-42d3-a341-7d407d0606c8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.793884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebc825-6c22-436d-b411-c9ede47e349b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.794641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebc825-6c22-436d-b411-c9ede47e349b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.794779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebc825-6c22-436d-b411-c9ede47e349b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.795042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebc825-6c22-436d-b411-c9ede47e349b admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.795682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebc825-6c22-436d-b411-c9ede47e349b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.797661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebc825-6c22-436d-b411-c9ede47e349b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.798193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c8b8b-b7f4-4bbf-a8f3-5e3ca22c80ec admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.804028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805961c3-b003-4535-b420-ea1c3fcbb199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.804789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805961c3-b003-4535-b420-ea1c3fcbb199 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.805196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805961c3-b003-4535-b420-ea1c3fcbb199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.805601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805961c3-b003-4535-b420-ea1c3fcbb199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.806432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805961c3-b003-4535-b420-ea1c3fcbb199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.809181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805961c3-b003-4535-b420-ea1c3fcbb199 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.821709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 938/1912] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:49.826906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805961c3-b003-4535-b420-ea1c3fcbb199 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.837288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805961c3-b003-4535-b420-ea1c3fcbb199 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.843152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9defb-0b0a-42c5-9903-02e052267f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.843828 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9defb-0b0a-42c5-9903-02e052267f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.844093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9defb-0b0a-42c5-9903-02e052267f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.844363 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9defb-0b0a-42c5-9903-02e052267f63 admin admin] PATH_INFO: `/v3/projects/dc08bd059e9e417e9b50c35c75bab782/users/f9fd95ec0790458a8bd27d7a1f88a9a8/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.861751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9defb-0b0a-42c5-9903-02e052267f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc08bd059e9e417e9b50c35c75bab782, user_id=f9fd95ec0790458a8bd27d7a1f88a9a8, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.864576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9defb-0b0a-42c5-9903-02e052267f63 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.913235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 975/1913] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:49 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:49.914242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebc825-6c22-436d-b411-c9ede47e349b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.924627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebc825-6c22-436d-b411-c9ede47e349b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.929354 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e549fa1-f4a4-41bf-943f-1c94dba8a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.929985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e549fa1-f4a4-41bf-943f-1c94dba8a2df admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.930264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e549fa1-f4a4-41bf-943f-1c94dba8a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.930538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e549fa1-f4a4-41bf-943f-1c94dba8a2df admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.931581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e549fa1-f4a4-41bf-943f-1c94dba8a2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.933120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e549fa1-f4a4-41bf-943f-1c94dba8a2df admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.943143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 939/1914] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:49 2022] PUT /identity/v3/projects/dc08bd059e9e417e9b50c35c75bab782/users/f9fd95ec0790458a8bd27d7a1f88a9a8/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:49.947353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9defb-0b0a-42c5-9903-02e052267f63 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.952277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 976/1915] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:49.956262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e549fa1-f4a4-41bf-943f-1c94dba8a2df admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.966333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e549fa1-f4a4-41bf-943f-1c94dba8a2df admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:50.015222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9defb-0b0a-42c5-9903-02e052267f63 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:50.020309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cc895-b262-44b7-b726-ab05164db95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:50.020952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cc895-b262-44b7-b726-ab05164db95c admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:50.021286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cc895-b262-44b7-b726-ab05164db95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:50.021593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cc895-b262-44b7-b726-ab05164db95c admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:50.022238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51cc895-b262-44b7-b726-ab05164db95c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:50.024006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51cc895-b262-44b7-b726-ab05164db95c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:50.038268 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 940/1916] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:50.040468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cc895-b262-44b7-b726-ab05164db95c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:50.050781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fad080-1029-4b89-8837-64f568114fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:50.054140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fad080-1029-4b89-8837-64f568114fed admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:50.054428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fad080-1029-4b89-8837-64f568114fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:50.054752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fad080-1029-4b89-8837-64f568114fed admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:50.055468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fad080-1029-4b89-8837-64f568114fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:50.057483 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fad080-1029-4b89-8837-64f568114fed admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:50.058395 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cc895-b262-44b7-b726-ab05164db95c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:50.064565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09482434-e547-4a8e-a384-52b4707ee7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:50.065284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09482434-e547-4a8e-a384-52b4707ee7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:50.065612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09482434-e547-4a8e-a384-52b4707ee7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:50.065923 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09482434-e547-4a8e-a384-52b4707ee7f1 admin admin] PATH_INFO: `/v3/projects/dc08bd059e9e417e9b50c35c75bab782/users/f9fd95ec0790458a8bd27d7a1f88a9a8/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:50.069729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09482434-e547-4a8e-a384-52b4707ee7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc08bd059e9e417e9b50c35c75bab782, user_id=f9fd95ec0790458a8bd27d7a1f88a9a8, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:50.072409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09482434-e547-4a8e-a384-52b4707ee7f1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:50.081139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 977/1917] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:50.082897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fad080-1029-4b89-8837-64f568114fed admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:50.093429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 941/1918] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:50 2022] PUT /identity/v3/projects/dc08bd059e9e417e9b50c35c75bab782/users/f9fd95ec0790458a8bd27d7a1f88a9a8/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:50.096758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fad080-1029-4b89-8837-64f568114fed admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:50.103530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146be58f-e1ec-4662-936c-ec78e31e6092 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:50.103831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146be58f-e1ec-4662-936c-ec78e31e6092 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:50.104088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146be58f-e1ec-4662-936c-ec78e31e6092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:50.135505 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-146be58f-e1ec-4662-936c-ec78e31e6092 None None] MFA Rules not processed for user `f9fd95ec0790458a8bd27d7a1f88a9a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:50.137427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-146be58f-e1ec-4662-936c-ec78e31e6092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:50.189007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f0989-8895-4ee6-9beb-a9c8e942fe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:50.189784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f0989-8895-4ee6-9beb-a9c8e942fe90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:50.190069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f0989-8895-4ee6-9beb-a9c8e942fe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:50.190372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f0989-8895-4ee6-9beb-a9c8e942fe90 admin admin] PATH_INFO: `/v3/projects/ff89d3eba9ae46cabc56a0a8d3d867a1/users/1ba7119871bd4bbcb1ed4d9ec6a1db87/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:50.198471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:50.211204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f0989-8895-4ee6-9beb-a9c8e942fe90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff89d3eba9ae46cabc56a0a8d3d867a1, user_id=1ba7119871bd4bbcb1ed4d9ec6a1db87, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:50.214234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f0989-8895-4ee6-9beb-a9c8e942fe90 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:50.226092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 942/1919] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:50.239371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146be58f-e1ec-4662-936c-ec78e31e6092 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:50.241255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-146be58f-e1ec-4662-936c-ec78e31e6092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:50.276648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 978/1920] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:50 2022] PUT /identity/v3/projects/ff89d3eba9ae46cabc56a0a8d3d867a1/users/1ba7119871bd4bbcb1ed4d9ec6a1db87/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:50.288996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa88182-0a85-43e6-a822-710f9abb922c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:50.289353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa88182-0a85-43e6-a822-710f9abb922c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:50.289763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa88182-0a85-43e6-a822-710f9abb922c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:50.310956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146be58f-e1ec-4662-936c-ec78e31e6092 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:50.320337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9fa88182-0a85-43e6-a822-710f9abb922c None None] MFA Rules not processed for user `1ba7119871bd4bbcb1ed4d9ec6a1db87`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:50.321678 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9fa88182-0a85-43e6-a822-710f9abb922c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:50.356960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:50.362916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dc1d0-67ea-4063-8035-a6631020f3e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:50.363537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dc1d0-67ea-4063-8035-a6631020f3e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:50.363848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dc1d0-67ea-4063-8035-a6631020f3e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:50.364121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dc1d0-67ea-4063-8035-a6631020f3e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:50.366324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-596dc1d0-67ea-4063-8035-a6631020f3e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:50.392779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 979/1921] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:50.405197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa88182-0a85-43e6-a822-710f9abb922c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:50.409333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9fa88182-0a85-43e6-a822-710f9abb922c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:50.448223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dc1d0-67ea-4063-8035-a6631020f3e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:50.450629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dc1d0-67ea-4063-8035-a6631020f3e0 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:50.463973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 943/1922] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:50.509965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa88182-0a85-43e6-a822-710f9abb922c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:50.514575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abc500-8a2f-4cac-83ca-f71b7d6d983b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c23597e0da2454abd2a5354d779b092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:50.515325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc500-8a2f-4cac-83ca-f71b7d6d983b service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:50.515611 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc500-8a2f-4cac-83ca-f71b7d6d983b service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:50.515891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc500-8a2f-4cac-83ca-f71b7d6d983b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:50.529669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abc500-8a2f-4cac-83ca-f71b7d6d983b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:50.531299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abc500-8a2f-4cac-83ca-f71b7d6d983b service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:50.543225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 980/1923] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:51.307639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dc1d0-67ea-4063-8035-a6631020f3e0 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:51.310672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-596dc1d0-67ea-4063-8035-a6631020f3e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:51.407261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:51.413414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dc1d0-67ea-4063-8035-a6631020f3e0 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:51.420238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:51.421542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:51.421823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:51.422102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] PATH_INFO: `/v3/users/9c90b89c710840c9951722a8432d620e` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:51.423920 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c90b89c710840c9951722a8432d620e)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:51.426592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:51.670518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c90b89c710840c9951722a8432d620e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.671260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c90b89c710840c9951722a8432d620e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.762636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c90b89c710840c9951722a8432d620e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.775524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c90b89c710840c9951722a8432d620e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.783572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c90b89c710840c9951722a8432d620e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.792552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 944/1924] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:51 2022] DELETE /identity/v3/users/9c90b89c710840c9951722a8432d620e => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:51.794661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31119172-45b1-4534-bd56-7bdc7c6746eb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:51.795360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31119172-45b1-4534-bd56-7bdc7c6746eb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:51.795595 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31119172-45b1-4534-bd56-7bdc7c6746eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:51.815671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-31119172-45b1-4534-bd56-7bdc7c6746eb None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:51.816937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-31119172-45b1-4534-bd56-7bdc7c6746eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:51.876477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 981/1925] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:51.886259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:51.888284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:51.947016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b403763-33b1-4521-91c7-1c21b57865a8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:51.951466 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0d107-5752-445e-8fb3-8b9d5c33fcb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:51.952075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0d107-5752-445e-8fb3-8b9d5c33fcb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:51.952358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0d107-5752-445e-8fb3-8b9d5c33fcb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:51.952630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0d107-5752-445e-8fb3-8b9d5c33fcb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:51.961823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0d107-5752-445e-8fb3-8b9d5c33fcb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:51.963263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0d107-5752-445e-8fb3-8b9d5c33fcb8 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:51.972743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 945/1926] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:52.554819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31119172-45b1-4534-bd56-7bdc7c6746eb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:52.556303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-31119172-45b1-4534-bd56-7bdc7c6746eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:52.570151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31119172-45b1-4534-bd56-7bdc7c6746eb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:52.576299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:52.576960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.577216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.579071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] PATH_INFO: `/v3/projects/aa8f493817594828931b4c8fe7d3cb93` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.585358 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8f493817594828931b4c8fe7d3cb93)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:52.590810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:52.752040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:52.753667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8f493817594828931b4c8fe7d3cb93', 'request_id': 'req-91127568-01b7-46ef-8bc4-e16568acc294'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:52.844174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8f493817594828931b4c8fe7d3cb93', 'request_id': 'req-91127568-01b7-46ef-8bc4-e16568acc294'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:52.853976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8f493817594828931b4c8fe7d3cb93', 'request_id': 'req-91127568-01b7-46ef-8bc4-e16568acc294'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:52.865564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] The token cache is being invalidate because project aa8f493817594828931b4c8fe7d3cb93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:52.865858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:52.866524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:52.868033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 982/1927] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:52 2022] DELETE /identity/v3/projects/aa8f493817594828931b4c8fe7d3cb93 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:52.872264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0d107-5752-445e-8fb3-8b9d5c33fcb8 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:52.881916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0d107-5752-445e-8fb3-8b9d5c33fcb8 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:52.948716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:52.949602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.949887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.950162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] PATH_INFO: `/v3/users/3b29c591b19e47c8be6a86b315bf41a9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.951296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b29c591b19e47c8be6a86b315bf41a9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:52.953316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:52.974811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:53.157808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b29c591b19e47c8be6a86b315bf41a9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:53.158458 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b29c591b19e47c8be6a86b315bf41a9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:53.224368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b29c591b19e47c8be6a86b315bf41a9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:53.237224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b29c591b19e47c8be6a86b315bf41a9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:53.244011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b29c591b19e47c8be6a86b315bf41a9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:53.250088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 946/1928] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:52 2022] DELETE /identity/v3/users/3b29c591b19e47c8be6a86b315bf41a9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:53.818475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:53.820000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:53.903791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91127568-01b7-46ef-8bc4-e16568acc294 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:53.909551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:53.911791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:54.031024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788bbe1-7f1e-4ea3-a759-3af95bf1824f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.031717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788bbe1-7f1e-4ea3-a759-3af95bf1824f service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.032002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788bbe1-7f1e-4ea3-a759-3af95bf1824f service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.032278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788bbe1-7f1e-4ea3-a759-3af95bf1824f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.032494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0550-693e-45ac-b546-5d85e6f2dc8b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.035027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f788bbe1-7f1e-4ea3-a759-3af95bf1824f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:54.045051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.046959 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.047430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.048154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] PATH_INFO: `/v3/projects/59866ccbc2a04bc7b471dd74f69cc343` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.060330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=59866ccbc2a04bc7b471dd74f69cc343)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.063503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.135651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788bbe1-7f1e-4ea3-a759-3af95bf1824f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.138104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788bbe1-7f1e-4ea3-a759-3af95bf1824f service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.149802 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 983/1929] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:54.247348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59866ccbc2a04bc7b471dd74f69cc343', 'request_id': 'req-f3876cea-ff66-482e-abb6-860424421aeb'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:54.323830 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59866ccbc2a04bc7b471dd74f69cc343', 'request_id': 'req-f3876cea-ff66-482e-abb6-860424421aeb'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:54.330756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59866ccbc2a04bc7b471dd74f69cc343', 'request_id': 'req-f3876cea-ff66-482e-abb6-860424421aeb'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:54.341802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] The token cache is being invalidate because project 59866ccbc2a04bc7b471dd74f69cc343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:54.342086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:54.342725 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3876cea-ff66-482e-abb6-860424421aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:54.344215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 947/1930] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:53 2022] DELETE /identity/v3/projects/59866ccbc2a04bc7b471dd74f69cc343 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:54.349214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa8b46-2f4d-42c7-9e9e-8553f7c1e1c0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.349539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa8b46-2f4d-42c7-9e9e-8553f7c1e1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.349839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa8b46-2f4d-42c7-9e9e-8553f7c1e1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.370328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-94aa8b46-2f4d-42c7-9e9e-8553f7c1e1c0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:54.372642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-94aa8b46-2f4d-42c7-9e9e-8553f7c1e1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:54.418642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:54.437053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 984/1931] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:54.446360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25534026-23f5-4a40-9809-6f332605c533 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.446649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25534026-23f5-4a40-9809-6f332605c533 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.446927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25534026-23f5-4a40-9809-6f332605c533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.468030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-25534026-23f5-4a40-9809-6f332605c533 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:54.469162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-25534026-23f5-4a40-9809-6f332605c533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:54.477193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 948/1932] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:54.481687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa8b46-2f4d-42c7-9e9e-8553f7c1e1c0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.491842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa8b46-2f4d-42c7-9e9e-8553f7c1e1c0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.497861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fc74e-169c-4a58-b43c-348972c51edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.498515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fc74e-169c-4a58-b43c-348972c51edd admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.498792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fc74e-169c-4a58-b43c-348972c51edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.499060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fc74e-169c-4a58-b43c-348972c51edd admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.499718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fc74e-169c-4a58-b43c-348972c51edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:54.500016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fc74e-169c-4a58-b43c-348972c51edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.501638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fc74e-169c-4a58-b43c-348972c51edd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.516290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 985/1933] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:31:54.518894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25534026-23f5-4a40-9809-6f332605c533 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.528669 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25534026-23f5-4a40-9809-6f332605c533 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.533498 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70944336-b82b-4814-84db-ace1644e8e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.534124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944336-b82b-4814-84db-ace1644e8e2d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.534402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944336-b82b-4814-84db-ace1644e8e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.534674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944336-b82b-4814-84db-ace1644e8e2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.535301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70944336-b82b-4814-84db-ace1644e8e2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.537311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70944336-b82b-4814-84db-ace1644e8e2d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.614121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:54.626037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 949/1934] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:54 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:54.627217 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fc74e-169c-4a58-b43c-348972c51edd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.637635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fc74e-169c-4a58-b43c-348972c51edd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.702151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f05c9-5333-49f3-a2cc-51166a748220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.702848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f05c9-5333-49f3-a2cc-51166a748220 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.703137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f05c9-5333-49f3-a2cc-51166a748220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.703408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f05c9-5333-49f3-a2cc-51166a748220 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.704047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f05c9-5333-49f3-a2cc-51166a748220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.707263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f05c9-5333-49f3-a2cc-51166a748220 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.806357 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 986/1935] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:54 2022] POST /identity/v3/users => generated 284 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:54.812752 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70944336-b82b-4814-84db-ace1644e8e2d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.827264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70944336-b82b-4814-84db-ace1644e8e2d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.831853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717b535-3660-4da9-999c-ebe94b3241ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.833327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717b535-3660-4da9-999c-ebe94b3241ea admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.833813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717b535-3660-4da9-999c-ebe94b3241ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.834168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717b535-3660-4da9-999c-ebe94b3241ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.834902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717b535-3660-4da9-999c-ebe94b3241ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.837022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717b535-3660-4da9-999c-ebe94b3241ea admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.862217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 950/1936] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:54.867009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f05c9-5333-49f3-a2cc-51166a748220 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.878910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f05c9-5333-49f3-a2cc-51166a748220 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.883945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c642857-f5d8-4fd0-b318-a0d13cde5df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.884653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c642857-f5d8-4fd0-b318-a0d13cde5df7 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.885274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c642857-f5d8-4fd0-b318-a0d13cde5df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.885777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c642857-f5d8-4fd0-b318-a0d13cde5df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.887161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c642857-f5d8-4fd0-b318-a0d13cde5df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.889142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c642857-f5d8-4fd0-b318-a0d13cde5df7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.900749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 987/1937] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:54.906012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717b535-3660-4da9-999c-ebe94b3241ea admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.917000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717b535-3660-4da9-999c-ebe94b3241ea admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.922558 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01706a7b-9f67-498b-bfd4-e57d3bf9d980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.923293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01706a7b-9f67-498b-bfd4-e57d3bf9d980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.923529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01706a7b-9f67-498b-bfd4-e57d3bf9d980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.923803 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01706a7b-9f67-498b-bfd4-e57d3bf9d980 admin admin] PATH_INFO: `/v3/projects/090d05c4239348aca372be6bc3b8eb15/users/5173600970e94f3195b7f8588770f6a2/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.940147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01706a7b-9f67-498b-bfd4-e57d3bf9d980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=090d05c4239348aca372be6bc3b8eb15, user_id=5173600970e94f3195b7f8588770f6a2, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.942908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01706a7b-9f67-498b-bfd4-e57d3bf9d980 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:55.030805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 951/1938] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:54 2022] PUT /identity/v3/projects/090d05c4239348aca372be6bc3b8eb15/users/5173600970e94f3195b7f8588770f6a2/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:55.041359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42305c-0c5b-4ea8-b380-a9b5ec585cfc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.041900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42305c-0c5b-4ea8-b380-a9b5ec585cfc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.042018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42305c-0c5b-4ea8-b380-a9b5ec585cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.073030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8c42305c-0c5b-4ea8-b380-a9b5ec585cfc None None] MFA Rules not processed for user `5173600970e94f3195b7f8588770f6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:55.074671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8c42305c-0c5b-4ea8-b380-a9b5ec585cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:55.138560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 988/1939] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:55.143052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01706a7b-9f67-498b-bfd4-e57d3bf9d980 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:55.205203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01706a7b-9f67-498b-bfd4-e57d3bf9d980 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:55.209903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7ed5d-a764-4849-a7a3-f78af4913b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:55.210504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7ed5d-a764-4849-a7a3-f78af4913b47 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.210784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7ed5d-a764-4849-a7a3-f78af4913b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.211058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7ed5d-a764-4849-a7a3-f78af4913b47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.211728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7ed5d-a764-4849-a7a3-f78af4913b47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:55.213674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7ed5d-a764-4849-a7a3-f78af4913b47 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:55.301393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 952/1940] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:55 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:55.303054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42305c-0c5b-4ea8-b380-a9b5ec585cfc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:55.316097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42305c-0c5b-4ea8-b380-a9b5ec585cfc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:55.359682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:55.377000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d4941-cd7a-4ec4-97d3-66e5f34ce387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:55.377651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d4941-cd7a-4ec4-97d3-66e5f34ce387 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.377933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d4941-cd7a-4ec4-97d3-66e5f34ce387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.378205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d4941-cd7a-4ec4-97d3-66e5f34ce387 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.378842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d4941-cd7a-4ec4-97d3-66e5f34ce387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:55.380785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d4941-cd7a-4ec4-97d3-66e5f34ce387 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:55.456376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 989/1941] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:55 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:55.457840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7ed5d-a764-4849-a7a3-f78af4913b47 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:55.472080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7ed5d-a764-4849-a7a3-f78af4913b47 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:55.477317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1300bf9-ccfa-48f1-8b08-04b268d86b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:55.478120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1300bf9-ccfa-48f1-8b08-04b268d86b76 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.478231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1300bf9-ccfa-48f1-8b08-04b268d86b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.478508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1300bf9-ccfa-48f1-8b08-04b268d86b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.479147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1300bf9-ccfa-48f1-8b08-04b268d86b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:55.481358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1300bf9-ccfa-48f1-8b08-04b268d86b76 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:55.493155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 953/1942] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:55.501231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d4941-cd7a-4ec4-97d3-66e5f34ce387 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:55.511339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d4941-cd7a-4ec4-97d3-66e5f34ce387 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:55.516068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970aa68-65fe-4e54-91ab-1724180558ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:55.516720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970aa68-65fe-4e54-91ab-1724180558ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.517003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970aa68-65fe-4e54-91ab-1724180558ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.517278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970aa68-65fe-4e54-91ab-1724180558ad admin admin] PATH_INFO: `/v3/projects/846174bb02b94f32adadd3c820a323ad/users/3038b3674faf4248a9bdeae796a8cb91/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.532124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970aa68-65fe-4e54-91ab-1724180558ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=846174bb02b94f32adadd3c820a323ad, user_id=3038b3674faf4248a9bdeae796a8cb91, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:55.534844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970aa68-65fe-4e54-91ab-1724180558ad admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:55.595585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 990/1943] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:55 2022] PUT /identity/v3/projects/846174bb02b94f32adadd3c820a323ad/users/3038b3674faf4248a9bdeae796a8cb91/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:55.599542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1300bf9-ccfa-48f1-8b08-04b268d86b76 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:55.609578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1300bf9-ccfa-48f1-8b08-04b268d86b76 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:55.667721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6fd42-b092-4ad4-839b-4d4081ca9981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:55.668330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6fd42-b092-4ad4-839b-4d4081ca9981 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.668604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6fd42-b092-4ad4-839b-4d4081ca9981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.668874 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6fd42-b092-4ad4-839b-4d4081ca9981 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.669588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6fd42-b092-4ad4-839b-4d4081ca9981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:55.671217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6fd42-b092-4ad4-839b-4d4081ca9981 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:55.681257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 954/1944] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:55.688303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970aa68-65fe-4e54-91ab-1724180558ad admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:55.701872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970aa68-65fe-4e54-91ab-1724180558ad admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:55.706536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81c66e-3258-4c97-b35c-b4e57000585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:55.707195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81c66e-3258-4c97-b35c-b4e57000585d admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.707475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81c66e-3258-4c97-b35c-b4e57000585d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.707749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81c66e-3258-4c97-b35c-b4e57000585d admin admin] PATH_INFO: `/v3/projects/846174bb02b94f32adadd3c820a323ad/users/3038b3674faf4248a9bdeae796a8cb91/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.709721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81c66e-3258-4c97-b35c-b4e57000585d admin admin] RBAC: Authorizing `identity:create_grant(project_id=846174bb02b94f32adadd3c820a323ad, user_id=3038b3674faf4248a9bdeae796a8cb91, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:55.716580 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81c66e-3258-4c97-b35c-b4e57000585d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:55.723648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 991/1945] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:55 2022] PUT /identity/v3/projects/846174bb02b94f32adadd3c820a323ad/users/3038b3674faf4248a9bdeae796a8cb91/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:55.737980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0393a90-86c4-40f5-928a-51e8ee45cccc None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.739154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0393a90-86c4-40f5-928a-51e8ee45cccc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.739436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0393a90-86c4-40f5-928a-51e8ee45cccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.768786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a0393a90-86c4-40f5-928a-51e8ee45cccc None None] MFA Rules not processed for user `3038b3674faf4248a9bdeae796a8cb91`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:55.770053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a0393a90-86c4-40f5-928a-51e8ee45cccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:55.789959 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:55.838294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 955/1946] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:31:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:55.851120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81c66e-3258-4c97-b35c-b4e57000585d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:55.852723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0b81c66e-3258-4c97-b35c-b4e57000585d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:55.915612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81c66e-3258-4c97-b35c-b4e57000585d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:55.919261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446e51b-119c-4387-8da4-56b53e545a56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:55.919871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446e51b-119c-4387-8da4-56b53e545a56 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.920150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446e51b-119c-4387-8da4-56b53e545a56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.920420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446e51b-119c-4387-8da4-56b53e545a56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.929349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446e51b-119c-4387-8da4-56b53e545a56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:55.930805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446e51b-119c-4387-8da4-56b53e545a56 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:55.941020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 992/1947] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:56.158296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0393a90-86c4-40f5-928a-51e8ee45cccc None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:56.168109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0393a90-86c4-40f5-928a-51e8ee45cccc None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:56.171702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab2caf-bb08-4b11-9cf5-b8f7d74ce458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:56.172312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab2caf-bb08-4b11-9cf5-b8f7d74ce458 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:56.172589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab2caf-bb08-4b11-9cf5-b8f7d74ce458 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:56.172860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab2caf-bb08-4b11-9cf5-b8f7d74ce458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:56.250597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab2caf-bb08-4b11-9cf5-b8f7d74ce458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:56.250597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab2caf-bb08-4b11-9cf5-b8f7d74ce458 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:56.256483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 956/1948] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:56.594023 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446e51b-119c-4387-8da4-56b53e545a56 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:56.595425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b446e51b-119c-4387-8da4-56b53e545a56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:56.606784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446e51b-119c-4387-8da4-56b53e545a56 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:56.611136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a554b0-f9df-4082-a906-09de26a05356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:56.611748 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a554b0-f9df-4082-a906-09de26a05356 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:56.612026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a554b0-f9df-4082-a906-09de26a05356 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:56.612294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a554b0-f9df-4082-a906-09de26a05356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:56.614129 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-04a554b0-f9df-4082-a906-09de26a05356 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:56.687757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a554b0-f9df-4082-a906-09de26a05356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:56.689298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a554b0-f9df-4082-a906-09de26a05356 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:56.701952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 993/1949] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:01.534296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab2caf-bb08-4b11-9cf5-b8f7d74ce458 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:01.608659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab2caf-bb08-4b11-9cf5-b8f7d74ce458 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:01.613377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:01.614067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:01.614344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:01.614639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] PATH_INFO: `/v3/users/5173600970e94f3195b7f8588770f6a2` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:01.615795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5173600970e94f3195b7f8588770f6a2)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:01.617830 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:01.800278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5173600970e94f3195b7f8588770f6a2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:01.801964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5173600970e94f3195b7f8588770f6a2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:01.877368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5173600970e94f3195b7f8588770f6a2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:01.882681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:01.892166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5173600970e94f3195b7f8588770f6a2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:01.899718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5173600970e94f3195b7f8588770f6a2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:01.906623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 957/1950] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:01 2022] DELETE /identity/v3/users/5173600970e94f3195b7f8588770f6a2 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:01.911291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d7eed-3f26-4eb3-b128-aa37652fba77 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:01.911827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d7eed-3f26-4eb3-b128-aa37652fba77 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:01.911827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d7eed-3f26-4eb3-b128-aa37652fba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:01.933588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7d0d7eed-3f26-4eb3-b128-aa37652fba77 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:01.935392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7d0d7eed-3f26-4eb3-b128-aa37652fba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:01.962340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:01.996042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 994/1951] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:02.005745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:02.007128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:02.070602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5eaec7-46c3-471c-984f-db7d45ad2d30 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:02.079857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa651d9-4ee6-4a87-adaf-4590911faf1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:02.081279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa651d9-4ee6-4a87-adaf-4590911faf1e service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:02.081279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa651d9-4ee6-4a87-adaf-4590911faf1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:02.081279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa651d9-4ee6-4a87-adaf-4590911faf1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:02.092562 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa651d9-4ee6-4a87-adaf-4590911faf1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:02.094024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa651d9-4ee6-4a87-adaf-4590911faf1e service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:02.103765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 958/1952] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:32:02.829128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d7eed-3f26-4eb3-b128-aa37652fba77 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:02.830749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7d0d7eed-3f26-4eb3-b128-aa37652fba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:02.849481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d7eed-3f26-4eb3-b128-aa37652fba77 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:02.863744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:02.864578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:02.864879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:02.865177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] PATH_INFO: `/v3/projects/090d05c4239348aca372be6bc3b8eb15` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:02.866573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=090d05c4239348aca372be6bc3b8eb15)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:02.868596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:03.047874 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090d05c4239348aca372be6bc3b8eb15', 'request_id': 'req-b8614aca-3c18-45fd-bc3c-4dec3da7f668'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:03.157547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090d05c4239348aca372be6bc3b8eb15', 'request_id': 'req-b8614aca-3c18-45fd-bc3c-4dec3da7f668'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:03.164412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090d05c4239348aca372be6bc3b8eb15', 'request_id': 'req-b8614aca-3c18-45fd-bc3c-4dec3da7f668'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:03.175271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] The token cache is being invalidate because project 090d05c4239348aca372be6bc3b8eb15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:03.175573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:03.176193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:03.177715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 995/1953] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:02 2022] DELETE /identity/v3/projects/090d05c4239348aca372be6bc3b8eb15 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:03.181698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa651d9-4ee6-4a87-adaf-4590911faf1e service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:03.193286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa651d9-4ee6-4a87-adaf-4590911faf1e service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:03.253634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:03.254211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:03.254496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:03.254774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] PATH_INFO: `/v3/users/3038b3674faf4248a9bdeae796a8cb91` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:03.256088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3038b3674faf4248a9bdeae796a8cb91)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:03.258249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:03.502550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3038b3674faf4248a9bdeae796a8cb91'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:03.503493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3038b3674faf4248a9bdeae796a8cb91'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:03.563742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3038b3674faf4248a9bdeae796a8cb91'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:03.583513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3038b3674faf4248a9bdeae796a8cb91'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:03.597613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3038b3674faf4248a9bdeae796a8cb91'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:03.599676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:03.609173 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 959/1954] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:03 2022] DELETE /identity/v3/users/3038b3674faf4248a9bdeae796a8cb91 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:03.669939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:03.671689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:03.694222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8614aca-3c18-45fd-bc3c-4dec3da7f668 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:03.835509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:03.835509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:03.835509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:03.835509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] PATH_INFO: `/v3/users/156b62ff74ba429b88e22c0b47595827` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:03.838003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=156b62ff74ba429b88e22c0b47595827)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:03.840901 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:03.983858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:03.985326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:04.009923 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7397512-1cff-49e0-871a-7e145db33ee3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:04.016610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:04.017362 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:04.020122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:04.020122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] PATH_INFO: `/v3/users/5fa12d4b835d4b59924f1ebec27f251b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:04.020122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fa12d4b835d4b59924f1ebec27f251b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:04.021182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:04.209534 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156b62ff74ba429b88e22c0b47595827'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.218374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:04.222382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fa12d4b835d4b59924f1ebec27f251b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.223326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fa12d4b835d4b59924f1ebec27f251b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.224076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156b62ff74ba429b88e22c0b47595827'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.233784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156b62ff74ba429b88e22c0b47595827'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.310433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fa12d4b835d4b59924f1ebec27f251b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.323871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fa12d4b835d4b59924f1ebec27f251b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.330935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fa12d4b835d4b59924f1ebec27f251b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.336982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 960/1955] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:03 2022] DELETE /identity/v3/users/5fa12d4b835d4b59924f1ebec27f251b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:04.337569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:04.339496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:04.363135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156b62ff74ba429b88e22c0b47595827'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.363755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ac4eb64f-735d-4ee6-a25a-82d9d8e309cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156b62ff74ba429b88e22c0b47595827'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.371021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 996/1956] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:03 2022] DELETE /identity/v3/users/156b62ff74ba429b88e22c0b47595827 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:04.372457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df7619-b73d-481b-956b-2dfe10638669 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:04.372969 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df7619-b73d-481b-956b-2dfe10638669 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:04.373556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df7619-b73d-481b-956b-2dfe10638669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:04.403394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c7df7619-b73d-481b-956b-2dfe10638669 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:04.404686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c7df7619-b73d-481b-956b-2dfe10638669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:04.407274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3408e-8235-45e2-ab9b-da887807797a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:04.463617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 997/1957] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:04.464908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d97e38-3aea-4fa5-8eb2-8d8903c82679 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:04.465197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d97e38-3aea-4fa5-8eb2-8d8903c82679 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:04.465488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d97e38-3aea-4fa5-8eb2-8d8903c82679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:04.468447 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:04.469118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:04.469354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:04.469645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] PATH_INFO: `/v3/projects/846174bb02b94f32adadd3c820a323ad` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:04.470765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=846174bb02b94f32adadd3c820a323ad)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:04.473513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:04.486395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-50d97e38-3aea-4fa5-8eb2-8d8903c82679 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:04.487742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-50d97e38-3aea-4fa5-8eb2-8d8903c82679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:04.496625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 998/1958] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:04.497615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d97e38-3aea-4fa5-8eb2-8d8903c82679 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:04.498963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-50d97e38-3aea-4fa5-8eb2-8d8903c82679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:04.510518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:04.556725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d97e38-3aea-4fa5-8eb2-8d8903c82679 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:04.561477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a00dad-de1f-4938-a1f9-453af081adb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:04.562100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a00dad-de1f-4938-a1f9-453af081adb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:04.562378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a00dad-de1f-4938-a1f9-453af081adb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:04.562650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a00dad-de1f-4938-a1f9-453af081adb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:04.571606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a00dad-de1f-4938-a1f9-453af081adb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:04.573032 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a00dad-de1f-4938-a1f9-453af081adb7 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:04.583630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 999/1959] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:04.584403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a00dad-de1f-4938-a1f9-453af081adb7 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:04.593499 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a00dad-de1f-4938-a1f9-453af081adb7 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:04.597130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd160dd-5b34-44f5-b6d2-b1ca4bc27379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:04.597777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd160dd-5b34-44f5-b6d2-b1ca4bc27379 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:04.598096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd160dd-5b34-44f5-b6d2-b1ca4bc27379 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:04.598325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd160dd-5b34-44f5-b6d2-b1ca4bc27379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:04.609908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd160dd-5b34-44f5-b6d2-b1ca4bc27379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:04.611376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd160dd-5b34-44f5-b6d2-b1ca4bc27379 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:04.619524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846174bb02b94f32adadd3c820a323ad', 'request_id': 'req-c773c24c-3651-49ec-97a0-57a8e946c055'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.622615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1000/1960] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:04.752899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846174bb02b94f32adadd3c820a323ad', 'request_id': 'req-c773c24c-3651-49ec-97a0-57a8e946c055'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.771503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846174bb02b94f32adadd3c820a323ad', 'request_id': 'req-c773c24c-3651-49ec-97a0-57a8e946c055'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.788442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] The token cache is being invalidate because project 846174bb02b94f32adadd3c820a323ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:04.788812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.789668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c773c24c-3651-49ec-97a0-57a8e946c055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:04.791184 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 961/1961] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:04 2022] DELETE /identity/v3/projects/846174bb02b94f32adadd3c820a323ad => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:04.797606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68e988-7336-498c-9d43-434c3ca69ed8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:04.797900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68e988-7336-498c-9d43-434c3ca69ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:04.798181 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68e988-7336-498c-9d43-434c3ca69ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:04.818839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-aa68e988-7336-498c-9d43-434c3ca69ed8 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:04.820126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-aa68e988-7336-498c-9d43-434c3ca69ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:04.881688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:04.900649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1001/1962] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:04.903740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9824d-c88c-433a-8c79-58f0ac2cc14e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:04.904031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9824d-c88c-433a-8c79-58f0ac2cc14e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:04.904483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9824d-c88c-433a-8c79-58f0ac2cc14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:04.934655 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c8c9824d-c88c-433a-8c79-58f0ac2cc14e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:04.936790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c8c9824d-c88c-433a-8c79-58f0ac2cc14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:04.948825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 962/1963] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:04.955275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68e988-7336-498c-9d43-434c3ca69ed8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:04.983364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68e988-7336-498c-9d43-434c3ca69ed8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:04.990925 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8803e-23c0-4243-accf-2736962a4612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:04.991564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8803e-23c0-4243-accf-2736962a4612 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:04.991842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8803e-23c0-4243-accf-2736962a4612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:04.992293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8803e-23c0-4243-accf-2736962a4612 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:04.993037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8803e-23c0-4243-accf-2736962a4612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:04.993348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8803e-23c0-4243-accf-2736962a4612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:04.995224 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8803e-23c0-4243-accf-2736962a4612 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:05.018619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1002/1964] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:32:05.020037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9824d-c88c-433a-8c79-58f0ac2cc14e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:05.038932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9824d-c88c-433a-8c79-58f0ac2cc14e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:05.048332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d8ea-352f-4f9b-bde6-cbc7672ca0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:05.049025 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d8ea-352f-4f9b-bde6-cbc7672ca0d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:05.049317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d8ea-352f-4f9b-bde6-cbc7672ca0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:05.050108 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d8ea-352f-4f9b-bde6-cbc7672ca0d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:05.050893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3d8ea-352f-4f9b-bde6-cbc7672ca0d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:05.052816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3d8ea-352f-4f9b-bde6-cbc7672ca0d1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:05.170193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 963/1965] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:05 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:05.174587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8803e-23c0-4243-accf-2736962a4612 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:05.186227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8803e-23c0-4243-accf-2736962a4612 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:05.303229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b50dc-5e01-48fb-85a2-fae0c2fb0941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:05.304749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b50dc-5e01-48fb-85a2-fae0c2fb0941 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:05.305218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b50dc-5e01-48fb-85a2-fae0c2fb0941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:05.306009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b50dc-5e01-48fb-85a2-fae0c2fb0941 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:05.307182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b50dc-5e01-48fb-85a2-fae0c2fb0941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:05.311441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b50dc-5e01-48fb-85a2-fae0c2fb0941 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:05.385608 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d8ea-352f-4f9b-bde6-cbc7672ca0d1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:05.389641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d3f3d8ea-352f-4f9b-bde6-cbc7672ca0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:05.406509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d8ea-352f-4f9b-bde6-cbc7672ca0d1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:05.420421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1003/1966] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:05 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:05.422753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:05.423630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:05.424045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:05.424185 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b50dc-5e01-48fb-85a2-fae0c2fb0941 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:05.424582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] PATH_INFO: `/v3/projects/5f2bd01b9c28466682470cbf168ee4ec` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:05.426263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2bd01b9c28466682470cbf168ee4ec)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:05.428408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:05.444021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b50dc-5e01-48fb-85a2-fae0c2fb0941 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:05.453752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c52f80-786b-496c-81a1-58a6d01fa850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:05.470247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c52f80-786b-496c-81a1-58a6d01fa850 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:05.470909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c52f80-786b-496c-81a1-58a6d01fa850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:05.471347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c52f80-786b-496c-81a1-58a6d01fa850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:05.472216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c52f80-786b-496c-81a1-58a6d01fa850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:05.474798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c52f80-786b-496c-81a1-58a6d01fa850 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:05.523483 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1004/1967] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:05.524379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c52f80-786b-496c-81a1-58a6d01fa850 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:05.526086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f3c52f80-786b-496c-81a1-58a6d01fa850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:05.545510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c52f80-786b-496c-81a1-58a6d01fa850 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:05.552625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:05.554063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:05.554063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:05.554063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] PATH_INFO: `/v3/projects/289727701c414a53917eb1de18fe1c5a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:05.555443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=289727701c414a53917eb1de18fe1c5a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:05.558013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:05.651798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2bd01b9c28466682470cbf168ee4ec', 'request_id': 'req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:05.722482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289727701c414a53917eb1de18fe1c5a', 'request_id': 'req-32bdc0d0-a513-494a-b376-af77b8e11e5b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:05.739954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2bd01b9c28466682470cbf168ee4ec', 'request_id': 'req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:05.742138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2bd01b9c28466682470cbf168ee4ec', 'request_id': 'req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:05.761981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] The token cache is being invalidate because project 5f2bd01b9c28466682470cbf168ee4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:05.762292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:05.767821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:05.769667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 964/1968] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:05 2022] DELETE /identity/v3/projects/5f2bd01b9c28466682470cbf168ee4ec => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:05.770268 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:05.771684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:05.844883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289727701c414a53917eb1de18fe1c5a', 'request_id': 'req-32bdc0d0-a513-494a-b376-af77b8e11e5b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:05.859326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289727701c414a53917eb1de18fe1c5a', 'request_id': 'req-32bdc0d0-a513-494a-b376-af77b8e11e5b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:05.874465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] The token cache is being invalidate because project 289727701c414a53917eb1de18fe1c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:05.880940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72305b75-8a5b-47fa-8d97-7cff8db8ec7d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:05.883793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:05.884947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:05.887100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1005/1969] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:05 2022] DELETE /identity/v3/projects/289727701c414a53917eb1de18fe1c5a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:05.890820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:05.892438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:05.892557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:05.967862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:05.977004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae68618-1a93-4a45-8018-780e54d3e8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:05.977753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae68618-1a93-4a45-8018-780e54d3e8df admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:05.978098 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae68618-1a93-4a45-8018-780e54d3e8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:05.978441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae68618-1a93-4a45-8018-780e54d3e8df admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:05.979861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae68618-1a93-4a45-8018-780e54d3e8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:05.982162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae68618-1a93-4a45-8018-780e54d3e8df admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:05.987801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdc0d0-a513-494a-b376-af77b8e11e5b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:05.993020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:05.993324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:05.993638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:05.993915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] PATH_INFO: `/v3/users/6f13845e1a984105a4848f953fe20a6b` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:05.995220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f13845e1a984105a4848f953fe20a6b)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:05.997212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:05.998210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 965/1970] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:05.999192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae68618-1a93-4a45-8018-780e54d3e8df admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:06.001908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7ae68618-1a93-4a45-8018-780e54d3e8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:06.015456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae68618-1a93-4a45-8018-780e54d3e8df admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:06.020534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:06.021170 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:06.021545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:06.021862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] PATH_INFO: `/v3/users/2bf635af33494c458ba399a3e354a951` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:06.023021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf635af33494c458ba399a3e354a951)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:06.024958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:06.282108 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf635af33494c458ba399a3e354a951'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:06.283112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf635af33494c458ba399a3e354a951'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:06.286625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f13845e1a984105a4848f953fe20a6b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:06.309336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f13845e1a984105a4848f953fe20a6b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:06.314242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f13845e1a984105a4848f953fe20a6b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:06.358062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf635af33494c458ba399a3e354a951'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:06.372177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf635af33494c458ba399a3e354a951'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:06.379354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf635af33494c458ba399a3e354a951'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:06.385726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 966/1971] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:05 2022] DELETE /identity/v3/users/2bf635af33494c458ba399a3e354a951 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:06.387341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:06.388802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:06.478039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f13845e1a984105a4848f953fe20a6b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:06.478755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d38189fc-4661-4a37-a5f7-c5ec4f426530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f13845e1a984105a4848f953fe20a6b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:06.490139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbefa5a-2a11-4a8d-9c95-5cbac11362f3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:06.493773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1006/1972] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:05 2022] DELETE /identity/v3/users/6f13845e1a984105a4848f953fe20a6b => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:06.498334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0318a-d4b3-44a0-a83e-2fd1aa360ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:06.498944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0318a-d4b3-44a0-a83e-2fd1aa360ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:06.499219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0318a-d4b3-44a0-a83e-2fd1aa360ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:06.499471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0318a-d4b3-44a0-a83e-2fd1aa360ea3 admin admin] PATH_INFO: `/v3/projects/b07ee43eda614ad4b71fbb9de5ae53fe/users/c0ac3676e9ea42b5a083d04d9244a4ed/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:06.520034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0318a-d4b3-44a0-a83e-2fd1aa360ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07ee43eda614ad4b71fbb9de5ae53fe, user_id=c0ac3676e9ea42b5a083d04d9244a4ed, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:06.522754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0318a-d4b3-44a0-a83e-2fd1aa360ea3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:06.596892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 967/1973] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:06 2022] PUT /identity/v3/projects/b07ee43eda614ad4b71fbb9de5ae53fe/users/c0ac3676e9ea42b5a083d04d9244a4ed/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:06.607743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9f1b-3d06-48c3-9f9b-481ac3d893ad None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:06.608658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9f1b-3d06-48c3-9f9b-481ac3d893ad None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:06.608658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c9f1b-3d06-48c3-9f9b-481ac3d893ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:06.648299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-738c9f1b-3d06-48c3-9f9b-481ac3d893ad None None] MFA Rules not processed for user `c0ac3676e9ea42b5a083d04d9244a4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:06.650435 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-738c9f1b-3d06-48c3-9f9b-481ac3d893ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:06.760146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1007/1974] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:06.765087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0318a-d4b3-44a0-a83e-2fd1aa360ea3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:06.766995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-07e0318a-d4b3-44a0-a83e-2fd1aa360ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:06.859489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0318a-d4b3-44a0-a83e-2fd1aa360ea3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:06.866998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4605d8-e59a-4159-b3ad-080a366ddf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:06.867682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4605d8-e59a-4159-b3ad-080a366ddf92 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:06.867961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4605d8-e59a-4159-b3ad-080a366ddf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:06.868233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4605d8-e59a-4159-b3ad-080a366ddf92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:06.868892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4605d8-e59a-4159-b3ad-080a366ddf92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:06.871849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4605d8-e59a-4159-b3ad-080a366ddf92 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:06.960067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c9f1b-3d06-48c3-9f9b-481ac3d893ad None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:06.961723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-738c9f1b-3d06-48c3-9f9b-481ac3d893ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:06.981776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c9f1b-3d06-48c3-9f9b-481ac3d893ad None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:06.985545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 968/1975] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:06 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:06.990745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4605d8-e59a-4159-b3ad-080a366ddf92 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:07.001560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:07.003676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.004618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.005174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] PATH_INFO: `/v3/projects/3324620b79f0449d86e5b5fbe666f028` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.007323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3324620b79f0449d86e5b5fbe666f028)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:07.011527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:07.061627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4605d8-e59a-4159-b3ad-080a366ddf92 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:07.066360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a906b-f45c-41b9-828b-996a6e77275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:07.066971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a906b-f45c-41b9-828b-996a6e77275b admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.067250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a906b-f45c-41b9-828b-996a6e77275b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.067522 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a906b-f45c-41b9-828b-996a6e77275b admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.068171 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a906b-f45c-41b9-828b-996a6e77275b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:07.070145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a906b-f45c-41b9-828b-996a6e77275b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:07.145950 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3324620b79f0449d86e5b5fbe666f028', 'request_id': 'req-ab2cd228-09be-4566-9db6-56415afb35b2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.190394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 969/1976] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:06 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:07.194473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a906b-f45c-41b9-828b-996a6e77275b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:07.196111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-448a906b-f45c-41b9-828b-996a6e77275b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:07.209266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a906b-f45c-41b9-828b-996a6e77275b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:07.212239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3324620b79f0449d86e5b5fbe666f028', 'request_id': 'req-ab2cd228-09be-4566-9db6-56415afb35b2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.219767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3324620b79f0449d86e5b5fbe666f028', 'request_id': 'req-ab2cd228-09be-4566-9db6-56415afb35b2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.241190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] The token cache is being invalidate because project 3324620b79f0449d86e5b5fbe666f028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:07.241731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.243064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.247570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1008/1977] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:06 2022] DELETE /identity/v3/projects/3324620b79f0449d86e5b5fbe666f028 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:07.260020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:07.260020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:07.281670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:07.282223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.282499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.282768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] PATH_INFO: `/v3/projects/80a5ea90efdf42039daaf1d4835dd79e` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.283893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a5ea90efdf42039daaf1d4835dd79e)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:07.285849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:07.329988 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cd228-09be-4566-9db6-56415afb35b2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:07.335187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fda86c-9809-4bd9-8a9f-34d594471622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:07.335187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fda86c-9809-4bd9-8a9f-34d594471622 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.335554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fda86c-9809-4bd9-8a9f-34d594471622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.335687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fda86c-9809-4bd9-8a9f-34d594471622 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.336315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fda86c-9809-4bd9-8a9f-34d594471622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:07.337982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fda86c-9809-4bd9-8a9f-34d594471622 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:07.348124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1009/1978] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:07.352282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fda86c-9809-4bd9-8a9f-34d594471622 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:07.354173 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-03fda86c-9809-4bd9-8a9f-34d594471622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:07.367741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fda86c-9809-4bd9-8a9f-34d594471622 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:07.373232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:07.373892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.374175 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.374447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] PATH_INFO: `/v3/users/f9fd95ec0790458a8bd27d7a1f88a9a8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.375711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9fd95ec0790458a8bd27d7a1f88a9a8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:07.377827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:07.405283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a5ea90efdf42039daaf1d4835dd79e', 'request_id': 'req-28d34f19-825f-4aac-95d9-886851587e40'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.423536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:07.509181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a5ea90efdf42039daaf1d4835dd79e', 'request_id': 'req-28d34f19-825f-4aac-95d9-886851587e40'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.518394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a5ea90efdf42039daaf1d4835dd79e', 'request_id': 'req-28d34f19-825f-4aac-95d9-886851587e40'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.531106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] The token cache is being invalidate because project 80a5ea90efdf42039daaf1d4835dd79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:07.531666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.532574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.534081 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 970/1979] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:07 2022] DELETE /identity/v3/projects/80a5ea90efdf42039daaf1d4835dd79e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:07.538398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:07.539720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:07.582085 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9fd95ec0790458a8bd27d7a1f88a9a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.594570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9fd95ec0790458a8bd27d7a1f88a9a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.603485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9fd95ec0790458a8bd27d7a1f88a9a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.618366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d34f19-825f-4aac-95d9-886851587e40 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:07.634760 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d93472-4d71-43ff-9fc7-8250627d4762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:07.635232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d93472-4d71-43ff-9fc7-8250627d4762 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.636279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d93472-4d71-43ff-9fc7-8250627d4762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.637371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d93472-4d71-43ff-9fc7-8250627d4762 admin admin] PATH_INFO: `/v3/projects/c300d2b14f5247b2bd7ba6fd4f9d19d1/users/45ea22162da8457dbb20164a034db818/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.669258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d93472-4d71-43ff-9fc7-8250627d4762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c300d2b14f5247b2bd7ba6fd4f9d19d1, user_id=45ea22162da8457dbb20164a034db818, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:07.674093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d93472-4d71-43ff-9fc7-8250627d4762 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:07.721356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9fd95ec0790458a8bd27d7a1f88a9a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.722039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eef604ab-db85-411b-82e6-8110b83c5ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9fd95ec0790458a8bd27d7a1f88a9a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.729387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1010/1980] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:07 2022] DELETE /identity/v3/users/f9fd95ec0790458a8bd27d7a1f88a9a8 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:07.731288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e2752-97c1-408a-b795-8bd1220ed0ef None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.731638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e2752-97c1-408a-b795-8bd1220ed0ef None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.735995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e2752-97c1-408a-b795-8bd1220ed0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.755292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-662e2752-97c1-408a-b795-8bd1220ed0ef None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:07.774044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 971/1981] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:07 2022] PUT /identity/v3/projects/c300d2b14f5247b2bd7ba6fd4f9d19d1/users/45ea22162da8457dbb20164a034db818/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:07.776199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:07.778588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d93472-4d71-43ff-9fc7-8250627d4762 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:07.779023 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-662e2752-97c1-408a-b795-8bd1220ed0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:07.794727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-43d93472-4d71-43ff-9fc7-8250627d4762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:07.885796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1011/1982] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:07.894048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac115ab7-81f9-4119-83f2-036e5525b086 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.894329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac115ab7-81f9-4119-83f2-036e5525b086 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.894603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac115ab7-81f9-4119-83f2-036e5525b086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.919887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d93472-4d71-43ff-9fc7-8250627d4762 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:07.924096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ac115ab7-81f9-4119-83f2-036e5525b086 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:07.926390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ac115ab7-81f9-4119-83f2-036e5525b086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:07.930294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6c357-4de8-493c-a4ff-d9f00a002425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:07.932135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6c357-4de8-493c-a4ff-d9f00a002425 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.932833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6c357-4de8-493c-a4ff-d9f00a002425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.934024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6c357-4de8-493c-a4ff-d9f00a002425 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.934742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6c357-4de8-493c-a4ff-d9f00a002425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:07.937695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6c357-4de8-493c-a4ff-d9f00a002425 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:07.941434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1012/1983] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:07.942628 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac115ab7-81f9-4119-83f2-036e5525b086 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:07.955327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 972/1984] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:07.956262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6c357-4de8-493c-a4ff-d9f00a002425 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:07.973480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac115ab7-81f9-4119-83f2-036e5525b086 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:07.973480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e0b4b-1f1c-48e8-9e18-76fa7ab5c280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:07.973480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e0b4b-1f1c-48e8-9e18-76fa7ab5c280 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.973480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e0b4b-1f1c-48e8-9e18-76fa7ab5c280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.973941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e0b4b-1f1c-48e8-9e18-76fa7ab5c280 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.973941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e0b4b-1f1c-48e8-9e18-76fa7ab5c280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:07.973941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e0b4b-1f1c-48e8-9e18-76fa7ab5c280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:07.974162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6c357-4de8-493c-a4ff-d9f00a002425 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:07.974250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e0b4b-1f1c-48e8-9e18-76fa7ab5c280 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:07.977571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7089d0-2e11-4afc-8d36-ed9a225b2e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:07.984299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7089d0-2e11-4afc-8d36-ed9a225b2e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.984299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7089d0-2e11-4afc-8d36-ed9a225b2e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.984299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7089d0-2e11-4afc-8d36-ed9a225b2e3e admin admin] PATH_INFO: `/v3/projects/c300d2b14f5247b2bd7ba6fd4f9d19d1/users/45ea22162da8457dbb20164a034db818/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.984299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7089d0-2e11-4afc-8d36-ed9a225b2e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c300d2b14f5247b2bd7ba6fd4f9d19d1, user_id=45ea22162da8457dbb20164a034db818, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:07.984684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7089d0-2e11-4afc-8d36-ed9a225b2e3e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:07.987567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1013/1985] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:32:07.992356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e0b4b-1f1c-48e8-9e18-76fa7ab5c280 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:07.998006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 973/1986] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:07 2022] PUT /identity/v3/projects/c300d2b14f5247b2bd7ba6fd4f9d19d1/users/45ea22162da8457dbb20164a034db818/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:08.002882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e0b4b-1f1c-48e8-9e18-76fa7ab5c280 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.017636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e91715-032b-4390-bae7-b531a1e83040 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.018169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e91715-032b-4390-bae7-b531a1e83040 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.018541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e91715-032b-4390-bae7-b531a1e83040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.066078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-72e91715-032b-4390-bae7-b531a1e83040 None None] MFA Rules not processed for user `45ea22162da8457dbb20164a034db818`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:08.066078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-72e91715-032b-4390-bae7-b531a1e83040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:08.109686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd8b0c-8702-4428-8978-4a84b51cef85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.110315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd8b0c-8702-4428-8978-4a84b51cef85 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.110591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd8b0c-8702-4428-8978-4a84b51cef85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.110857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd8b0c-8702-4428-8978-4a84b51cef85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.111489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd8b0c-8702-4428-8978-4a84b51cef85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.113384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd8b0c-8702-4428-8978-4a84b51cef85 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.173544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 974/1987] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:08.188751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e91715-032b-4390-bae7-b531a1e83040 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:08.188751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-72e91715-032b-4390-bae7-b531a1e83040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:08.227295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1014/1988] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:07 2022] POST /identity/v3/projects => generated 363 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:08.232328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd8b0c-8702-4428-8978-4a84b51cef85 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:08.270759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e91715-032b-4390-bae7-b531a1e83040 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.321522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd8b0c-8702-4428-8978-4a84b51cef85 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.339642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccb497-c354-466f-8357-f08074224aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.340310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccb497-c354-466f-8357-f08074224aea admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.340690 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccb497-c354-466f-8357-f08074224aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.340973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccb497-c354-466f-8357-f08074224aea admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.341646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccb497-c354-466f-8357-f08074224aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.343867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccb497-c354-466f-8357-f08074224aea admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.366825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e6231-7e7c-4d8f-a473-72883aa9c905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.367619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e6231-7e7c-4d8f-a473-72883aa9c905 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.368009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e6231-7e7c-4d8f-a473-72883aa9c905 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.369877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e6231-7e7c-4d8f-a473-72883aa9c905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.447284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1015/1989] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:08 2022] POST /identity/v3/users => generated 282 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:08.450264 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccb497-c354-466f-8357-f08074224aea admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:08.452054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-36ccb497-c354-466f-8357-f08074224aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:08.465756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccb497-c354-466f-8357-f08074224aea admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.472342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.473149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.473425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.473749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] PATH_INFO: `/v3/projects/dc08bd059e9e417e9b50c35c75bab782` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.474963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc08bd059e9e417e9b50c35c75bab782)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.476885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.487854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e6231-7e7c-4d8f-a473-72883aa9c905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.490592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e6231-7e7c-4d8f-a473-72883aa9c905 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.525739 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 975/1990] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:32:08.526723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e6231-7e7c-4d8f-a473-72883aa9c905 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:08.549721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e6231-7e7c-4d8f-a473-72883aa9c905 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.570525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efbd0e-3341-4613-805e-955bf027008a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.570525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efbd0e-3341-4613-805e-955bf027008a admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.570525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efbd0e-3341-4613-805e-955bf027008a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.570525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efbd0e-3341-4613-805e-955bf027008a admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.570951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efbd0e-3341-4613-805e-955bf027008a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.572266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efbd0e-3341-4613-805e-955bf027008a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.596612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 976/1991] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:08.603280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efbd0e-3341-4613-805e-955bf027008a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:08.620140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efbd0e-3341-4613-805e-955bf027008a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.623076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc08bd059e9e417e9b50c35c75bab782', 'request_id': 'req-55f5726b-816e-498e-985f-03df936b870d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.688813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f513c1-852d-4940-af1c-b475f0494ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.689428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f513c1-852d-4940-af1c-b475f0494ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.689743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f513c1-852d-4940-af1c-b475f0494ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.690009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f513c1-852d-4940-af1c-b475f0494ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.690650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f513c1-852d-4940-af1c-b475f0494ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.692251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f513c1-852d-4940-af1c-b475f0494ae9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.698054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc08bd059e9e417e9b50c35c75bab782', 'request_id': 'req-55f5726b-816e-498e-985f-03df936b870d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.707994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc08bd059e9e417e9b50c35c75bab782', 'request_id': 'req-55f5726b-816e-498e-985f-03df936b870d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.710065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 977/1992] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:08.711706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f513c1-852d-4940-af1c-b475f0494ae9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:08.727289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] The token cache is being invalidate because project dc08bd059e9e417e9b50c35c75bab782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:08.727526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.728607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.730473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1016/1993] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:08 2022] DELETE /identity/v3/projects/dc08bd059e9e417e9b50c35c75bab782 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:08.731446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:08.732800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:08.733323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f513c1-852d-4940-af1c-b475f0494ae9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.739084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.739721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.740280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.740633 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] PATH_INFO: `/v3/users/c0ac3676e9ea42b5a083d04d9244a4ed` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.741947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ac3676e9ea42b5a083d04d9244a4ed)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.749119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.749565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5726b-816e-498e-985f-03df936b870d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.754578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cafd4-884e-4272-bf20-1ac919e2159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.755363 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cafd4-884e-4272-bf20-1ac919e2159a admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.755712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cafd4-884e-4272-bf20-1ac919e2159a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.756059 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cafd4-884e-4272-bf20-1ac919e2159a admin admin] PATH_INFO: `/v3/projects/5716551007a448408d043f7d84edb1b0/users/a9cf28e775ff4f6fa4a107522a53c184/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.773188 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cafd4-884e-4272-bf20-1ac919e2159a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5716551007a448408d043f7d84edb1b0, user_id=a9cf28e775ff4f6fa4a107522a53c184, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.775892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cafd4-884e-4272-bf20-1ac919e2159a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.868030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1017/1994] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:08 2022] PUT /identity/v3/projects/5716551007a448408d043f7d84edb1b0/users/a9cf28e775ff4f6fa4a107522a53c184/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:08.869414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54025a5-44fa-4389-a891-bb653a08fb5a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.869719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54025a5-44fa-4389-a891-bb653a08fb5a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.869992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54025a5-44fa-4389-a891-bb653a08fb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.892506 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a54025a5-44fa-4389-a891-bb653a08fb5a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:08.900792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a54025a5-44fa-4389-a891-bb653a08fb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:08.965718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ac3676e9ea42b5a083d04d9244a4ed'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.966987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ac3676e9ea42b5a083d04d9244a4ed'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.994180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1018/1995] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:08.996218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6740c-0e4b-4588-8158-ab0f122e69c5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.997033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6740c-0e4b-4588-8158-ab0f122e69c5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.997033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6740c-0e4b-4588-8158-ab0f122e69c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.035811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:09.042364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-90c6740c-0e4b-4588-8158-ab0f122e69c5 None None] MFA Rules not processed for user `a9cf28e775ff4f6fa4a107522a53c184`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:09.049648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-90c6740c-0e4b-4588-8158-ab0f122e69c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:09.064566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ac3676e9ea42b5a083d04d9244a4ed'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.092195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ac3676e9ea42b5a083d04d9244a4ed'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.095798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:09.106976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-098aef5c-faa7-437d-a5cb-3cfb068fb479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ac3676e9ea42b5a083d04d9244a4ed'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.114610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 978/1996] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:08 2022] DELETE /identity/v3/users/c0ac3676e9ea42b5a083d04d9244a4ed => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:09.115936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe39aa8-f99a-448b-9b3c-f9d6a2bc603b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.116284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe39aa8-f99a-448b-9b3c-f9d6a2bc603b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.121187 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe39aa8-f99a-448b-9b3c-f9d6a2bc603b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.166099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1019/1997] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:09.168148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42882a-cb68-4af5-8be9-342cb493a2b0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.168650 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42882a-cb68-4af5-8be9-342cb493a2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.169000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42882a-cb68-4af5-8be9-342cb493a2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.171915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ebe39aa8-f99a-448b-9b3c-f9d6a2bc603b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:09.174156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ebe39aa8-f99a-448b-9b3c-f9d6a2bc603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:09.197197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6b42882a-cb68-4af5-8be9-342cb493a2b0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:09.198640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6b42882a-cb68-4af5-8be9-342cb493a2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:09.276540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1020/1998] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:09.277854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42882a-cb68-4af5-8be9-342cb493a2b0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.279259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6b42882a-cb68-4af5-8be9-342cb493a2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:09.286138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 979/1999] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:09.287332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe39aa8-f99a-448b-9b3c-f9d6a2bc603b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.288781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ebe39aa8-f99a-448b-9b3c-f9d6a2bc603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:09.292784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42882a-cb68-4af5-8be9-342cb493a2b0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.299303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d29a3-4828-41c5-ad63-8fd972ebf6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.299999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d29a3-4828-41c5-ad63-8fd972ebf6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.300346 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d29a3-4828-41c5-ad63-8fd972ebf6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.300691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d29a3-4828-41c5-ad63-8fd972ebf6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.301349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d29a3-4828-41c5-ad63-8fd972ebf6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.303750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d29a3-4828-41c5-ad63-8fd972ebf6a3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.357144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe39aa8-f99a-448b-9b3c-f9d6a2bc603b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.358818 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:09.362526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8be09-dc7c-4d84-b14d-25b2a0df7c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.363242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8be09-dc7c-4d84-b14d-25b2a0df7c21 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.364666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8be09-dc7c-4d84-b14d-25b2a0df7c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.364666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8be09-dc7c-4d84-b14d-25b2a0df7c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.375620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8be09-dc7c-4d84-b14d-25b2a0df7c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.377237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8be09-dc7c-4d84-b14d-25b2a0df7c21 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.388521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 980/2000] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:32:09.390782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8be09-dc7c-4d84-b14d-25b2a0df7c21 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.400588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1021/2001] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:09 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:09.401613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d29a3-4828-41c5-ad63-8fd972ebf6a3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.404022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8be09-dc7c-4d84-b14d-25b2a0df7c21 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.487661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d29a3-4828-41c5-ad63-8fd972ebf6a3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.491526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e0469-3310-4b4c-812c-a3c8009dfb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.492856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e0469-3310-4b4c-812c-a3c8009dfb09 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.493298 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e0469-3310-4b4c-812c-a3c8009dfb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.493435 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5cc60-3bb9-42f8-9443-76b329fd5ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.493620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e0469-3310-4b4c-812c-a3c8009dfb09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.494017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5cc60-3bb9-42f8-9443-76b329fd5ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.494278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e0469-3310-4b4c-812c-a3c8009dfb09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:09.494392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5cc60-3bb9-42f8-9443-76b329fd5ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.494583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e0469-3310-4b4c-812c-a3c8009dfb09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.494712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5cc60-3bb9-42f8-9443-76b329fd5ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.495284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5cc60-3bb9-42f8-9443-76b329fd5ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.496249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e0469-3310-4b4c-812c-a3c8009dfb09 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.497235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5cc60-3bb9-42f8-9443-76b329fd5ae4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.511854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:09.517044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 981/2002] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:32:09.522826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e0469-3310-4b4c-812c-a3c8009dfb09 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.538069 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e0469-3310-4b4c-812c-a3c8009dfb09 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.543787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14a41e-6ece-4f86-aa32-6c64a382e65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.544776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14a41e-6ece-4f86-aa32-6c64a382e65f admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.545231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14a41e-6ece-4f86-aa32-6c64a382e65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.545532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14a41e-6ece-4f86-aa32-6c64a382e65f admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.546169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14a41e-6ece-4f86-aa32-6c64a382e65f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.548168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14a41e-6ece-4f86-aa32-6c64a382e65f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.647978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1022/2003] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:09 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:09.650070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5cc60-3bb9-42f8-9443-76b329fd5ae4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.652831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 982/2004] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:09 2022] POST /identity/v3/projects => generated 361 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:09.653831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14a41e-6ece-4f86-aa32-6c64a382e65f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.665242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5cc60-3bb9-42f8-9443-76b329fd5ae4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.735312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:09.777926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14a41e-6ece-4f86-aa32-6c64a382e65f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.777926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cb002-77d4-47eb-b763-63c18ecb7b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.777926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cb002-77d4-47eb-b763-63c18ecb7b76 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.777926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cb002-77d4-47eb-b763-63c18ecb7b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.778884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cb002-77d4-47eb-b763-63c18ecb7b76 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.778884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cb002-77d4-47eb-b763-63c18ecb7b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.778884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cb002-77d4-47eb-b763-63c18ecb7b76 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.783750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e833fa-bedc-46cb-98d2-c4b7ba9aa980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.785056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e833fa-bedc-46cb-98d2-c4b7ba9aa980 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.785056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e833fa-bedc-46cb-98d2-c4b7ba9aa980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.785306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e833fa-bedc-46cb-98d2-c4b7ba9aa980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.786291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e833fa-bedc-46cb-98d2-c4b7ba9aa980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.787891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e833fa-bedc-46cb-98d2-c4b7ba9aa980 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.805390 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1023/2005] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:09.806643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e833fa-bedc-46cb-98d2-c4b7ba9aa980 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.818865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e833fa-bedc-46cb-98d2-c4b7ba9aa980 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.824619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e281e1-de5a-4bb8-b55e-59c8ce8970b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.825520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e281e1-de5a-4bb8-b55e-59c8ce8970b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.825981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e281e1-de5a-4bb8-b55e-59c8ce8970b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.826421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e281e1-de5a-4bb8-b55e-59c8ce8970b5 admin admin] PATH_INFO: `/v3/projects/4577656f83bd4224952086482857116a/users/37dcef4a31af44e9b8dd0f6d7e5a4c44/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.848517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e281e1-de5a-4bb8-b55e-59c8ce8970b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4577656f83bd4224952086482857116a, user_id=37dcef4a31af44e9b8dd0f6d7e5a4c44, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.852562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e281e1-de5a-4bb8-b55e-59c8ce8970b5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.910517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 983/2006] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:09 2022] POST /identity/v3/users => generated 281 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:09.912498 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cb002-77d4-47eb-b763-63c18ecb7b76 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.924462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cb002-77d4-47eb-b763-63c18ecb7b76 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.932042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce835bcd-1c72-400d-9dbd-65475e30ab9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.932698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce835bcd-1c72-400d-9dbd-65475e30ab9d admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.933718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce835bcd-1c72-400d-9dbd-65475e30ab9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.933718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce835bcd-1c72-400d-9dbd-65475e30ab9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.940077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce835bcd-1c72-400d-9dbd-65475e30ab9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.940077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce835bcd-1c72-400d-9dbd-65475e30ab9d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.947158 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 984/2007] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:09.954331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce835bcd-1c72-400d-9dbd-65475e30ab9d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.969965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce835bcd-1c72-400d-9dbd-65475e30ab9d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.974925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f05bc4-5f90-47e1-90d9-80cb74d58b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.975542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f05bc4-5f90-47e1-90d9-80cb74d58b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.975829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f05bc4-5f90-47e1-90d9-80cb74d58b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.976154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f05bc4-5f90-47e1-90d9-80cb74d58b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.976804 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f05bc4-5f90-47e1-90d9-80cb74d58b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.978462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f05bc4-5f90-47e1-90d9-80cb74d58b58 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.993500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 985/2008] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:10.000043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f05bc4-5f90-47e1-90d9-80cb74d58b58 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.006291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1024/2009] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:09 2022] PUT /identity/v3/projects/4577656f83bd4224952086482857116a/users/37dcef4a31af44e9b8dd0f6d7e5a4c44/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:10.015867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e281e1-de5a-4bb8-b55e-59c8ce8970b5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.023917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f05bc4-5f90-47e1-90d9-80cb74d58b58 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.095399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:10.118317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4227f-937f-46bb-8637-2176d494340a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.119191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4227f-937f-46bb-8637-2176d494340a admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.119539 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4227f-937f-46bb-8637-2176d494340a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.119817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4227f-937f-46bb-8637-2176d494340a admin admin] PATH_INFO: `/v3/projects/0bff778f3b4646c88fad2c4b6ac98a93/users/9731ff85af1c4b5fa8b2167dd5ddc728/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.146130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4227f-937f-46bb-8637-2176d494340a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bff778f3b4646c88fad2c4b6ac98a93, user_id=9731ff85af1c4b5fa8b2167dd5ddc728, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.149346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4227f-937f-46bb-8637-2176d494340a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.164945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e281e1-de5a-4bb8-b55e-59c8ce8970b5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.173368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0971e-d021-40ad-9bda-7424da8e4f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.174480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0971e-d021-40ad-9bda-7424da8e4f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.175137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0971e-d021-40ad-9bda-7424da8e4f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.175137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0971e-d021-40ad-9bda-7424da8e4f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.175950 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0971e-d021-40ad-9bda-7424da8e4f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.177727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0971e-d021-40ad-9bda-7424da8e4f68 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.190142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1025/2010] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:10.194246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0971e-d021-40ad-9bda-7424da8e4f68 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.206516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0971e-d021-40ad-9bda-7424da8e4f68 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.213259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106a3c4-cb94-46b3-9b20-8fd9a322875b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.215243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106a3c4-cb94-46b3-9b20-8fd9a322875b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.215696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106a3c4-cb94-46b3-9b20-8fd9a322875b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.220426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106a3c4-cb94-46b3-9b20-8fd9a322875b admin admin] PATH_INFO: `/v3/projects/4577656f83bd4224952086482857116a/users/37dcef4a31af44e9b8dd0f6d7e5a4c44/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.220426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106a3c4-cb94-46b3-9b20-8fd9a322875b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4577656f83bd4224952086482857116a, user_id=37dcef4a31af44e9b8dd0f6d7e5a4c44, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.222147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106a3c4-cb94-46b3-9b20-8fd9a322875b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.237161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1026/2011] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:10 2022] PUT /identity/v3/projects/4577656f83bd4224952086482857116a/users/37dcef4a31af44e9b8dd0f6d7e5a4c44/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:10.247774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d50ab9-301d-41f1-83b3-fa4326f78726 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.248060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d50ab9-301d-41f1-83b3-fa4326f78726 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.248369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d50ab9-301d-41f1-83b3-fa4326f78726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.275257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 986/2012] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:09 2022] PUT /identity/v3/projects/0bff778f3b4646c88fad2c4b6ac98a93/users/9731ff85af1c4b5fa8b2167dd5ddc728/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:10.282277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b5d50ab9-301d-41f1-83b3-fa4326f78726 None None] MFA Rules not processed for user `37dcef4a31af44e9b8dd0f6d7e5a4c44`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:10.283790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b5d50ab9-301d-41f1-83b3-fa4326f78726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:10.284891 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4227f-937f-46bb-8637-2176d494340a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.292166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-70e4227f-937f-46bb-8637-2176d494340a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:10.295289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:10.387312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1027/2013] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:10.390393 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87823940-4569-4209-87e2-f6ff9bbb5503 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.391496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87823940-4569-4209-87e2-f6ff9bbb5503 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.391926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87823940-4569-4209-87e2-f6ff9bbb5503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.408100 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4227f-937f-46bb-8637-2176d494340a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.415560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.416322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.416686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.417969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] PATH_INFO: `/v3/projects/b07ee43eda614ad4b71fbb9de5ae53fe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.420404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b07ee43eda614ad4b71fbb9de5ae53fe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.423052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.430026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-87823940-4569-4209-87e2-f6ff9bbb5503 None None] MFA Rules not processed for user `9731ff85af1c4b5fa8b2167dd5ddc728`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:10.431986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-87823940-4569-4209-87e2-f6ff9bbb5503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:10.513090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1028/2014] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:10.514067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87823940-4569-4209-87e2-f6ff9bbb5503 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.515485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-87823940-4569-4209-87e2-f6ff9bbb5503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:10.584753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b07ee43eda614ad4b71fbb9de5ae53fe', 'request_id': 'req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:10.585494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87823940-4569-4209-87e2-f6ff9bbb5503 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.641527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82449e2d-bdaa-4ee0-8c88-9c99f4698ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.642058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82449e2d-bdaa-4ee0-8c88-9c99f4698ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.642335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82449e2d-bdaa-4ee0-8c88-9c99f4698ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.642632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82449e2d-bdaa-4ee0-8c88-9c99f4698ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.708449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b07ee43eda614ad4b71fbb9de5ae53fe', 'request_id': 'req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:10.723439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b07ee43eda614ad4b71fbb9de5ae53fe', 'request_id': 'req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:10.741445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82449e2d-bdaa-4ee0-8c88-9c99f4698ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.743218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82449e2d-bdaa-4ee0-8c88-9c99f4698ad9 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.749102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] The token cache is being invalidate because project b07ee43eda614ad4b71fbb9de5ae53fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:10.752239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:10.754426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:10.757820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 987/2015] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:10 2022] DELETE /identity/v3/projects/b07ee43eda614ad4b71fbb9de5ae53fe => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:10.758833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1029/2016] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:10.760245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.769697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82449e2d-bdaa-4ee0-8c88-9c99f4698ad9 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.770028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:10.854807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82449e2d-bdaa-4ee0-8c88-9c99f4698ad9 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.861772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.862584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.862967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.863339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] PATH_INFO: `/v3/users/45ea22162da8457dbb20164a034db818` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.865495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ea22162da8457dbb20164a034db818)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.868125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.897268 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fe4f8-1c2f-4ab5-9673-a8b28e680297 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.901272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ac5cd-2251-4056-bf37-7b9115b276ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c23597e0da2454abd2a5354d779b092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.902846 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ac5cd-2251-4056-bf37-7b9115b276ef service glance] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.903398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ac5cd-2251-4056-bf37-7b9115b276ef service glance] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.903681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ac5cd-2251-4056-bf37-7b9115b276ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.905683 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7a5ac5cd-2251-4056-bf37-7b9115b276ef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:10.968580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ac5cd-2251-4056-bf37-7b9115b276ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.970071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ac5cd-2251-4056-bf37-7b9115b276ef service glance] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.979751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 988/2017] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:11.037803 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ac5cd-2251-4056-bf37-7b9115b276ef service glance] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:11.039245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7a5ac5cd-2251-4056-bf37-7b9115b276ef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:11.061072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ac5cd-2251-4056-bf37-7b9115b276ef service glance] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:11.065256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dad620-f014-4748-8fbc-1455534c18a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:11.066080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad620-f014-4748-8fbc-1455534c18a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:11.066531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad620-f014-4748-8fbc-1455534c18a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:11.067952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad620-f014-4748-8fbc-1455534c18a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:11.082487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ea22162da8457dbb20164a034db818'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:11.090556 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f8dad620-f014-4748-8fbc-1455534c18a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:11.101040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ea22162da8457dbb20164a034db818'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:11.110608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ea22162da8457dbb20164a034db818'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:11.187552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dad620-f014-4748-8fbc-1455534c18a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:11.189664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dad620-f014-4748-8fbc-1455534c18a3 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:11.196394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ea22162da8457dbb20164a034db818'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:11.197177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8397c894-8fd3-45ce-9626-264e8b14997c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ea22162da8457dbb20164a034db818'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:11.202086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 989/2018] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:32:11.206124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1030/2019] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:10 2022] DELETE /identity/v3/users/45ea22162da8457dbb20164a034db818 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:11.662864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:11.737136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dad620-f014-4748-8fbc-1455534c18a3 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:11.738642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f8dad620-f014-4748-8fbc-1455534c18a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:11.766722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dad620-f014-4748-8fbc-1455534c18a3 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:11.918253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:11.919283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:11.919830 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:11.919994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:11.920523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] PATH_INFO: `/v3/projects/c300d2b14f5247b2bd7ba6fd4f9d19d1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:11.922865 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c300d2b14f5247b2bd7ba6fd4f9d19d1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:11.926185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.081394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c300d2b14f5247b2bd7ba6fd4f9d19d1', 'request_id': 'req-4d8318a0-d8a0-4965-9be6-ed425cfa4857'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:12.151379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c300d2b14f5247b2bd7ba6fd4f9d19d1', 'request_id': 'req-4d8318a0-d8a0-4965-9be6-ed425cfa4857'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:12.159694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c300d2b14f5247b2bd7ba6fd4f9d19d1', 'request_id': 'req-4d8318a0-d8a0-4965-9be6-ed425cfa4857'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:12.174292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] The token cache is being invalidate because project c300d2b14f5247b2bd7ba6fd4f9d19d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:12.174585 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:12.175217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4d8318a0-d8a0-4965-9be6-ed425cfa4857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:12.177038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 990/2020] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:11 2022] DELETE /identity/v3/projects/c300d2b14f5247b2bd7ba6fd4f9d19d1 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:12.182740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc8d17-1d67-4b9d-840f-0ddd4906eef0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.183996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc8d17-1d67-4b9d-840f-0ddd4906eef0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.184561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc8d17-1d67-4b9d-840f-0ddd4906eef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.213371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7ebc8d17-1d67-4b9d-840f-0ddd4906eef0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:12.215507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7ebc8d17-1d67-4b9d-840f-0ddd4906eef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:12.295391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1031/2021] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:12.297602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489f843-43c3-4d48-b4d5-a13d65cbe32b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.297891 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489f843-43c3-4d48-b4d5-a13d65cbe32b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.298164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489f843-43c3-4d48-b4d5-a13d65cbe32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.320275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6489f843-43c3-4d48-b4d5-a13d65cbe32b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:12.326005 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6489f843-43c3-4d48-b4d5-a13d65cbe32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:12.336862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 991/2022] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:12.337380 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc8d17-1d67-4b9d-840f-0ddd4906eef0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.349801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc8d17-1d67-4b9d-840f-0ddd4906eef0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.360463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994617f-85d1-4bca-82f8-ebc2d11bd57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.361100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994617f-85d1-4bca-82f8-ebc2d11bd57f admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.361381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994617f-85d1-4bca-82f8-ebc2d11bd57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.362430 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994617f-85d1-4bca-82f8-ebc2d11bd57f admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.363114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994617f-85d1-4bca-82f8-ebc2d11bd57f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:12.363596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994617f-85d1-4bca-82f8-ebc2d11bd57f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.365218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994617f-85d1-4bca-82f8-ebc2d11bd57f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.383546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1032/2023] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:32:12.388851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489f843-43c3-4d48-b4d5-a13d65cbe32b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.399063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489f843-43c3-4d48-b4d5-a13d65cbe32b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.404275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c8374-a5e7-420f-b0f9-f2bbe298e3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.406673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c8374-a5e7-420f-b0f9-f2bbe298e3d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.406673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c8374-a5e7-420f-b0f9-f2bbe298e3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.406673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c8374-a5e7-420f-b0f9-f2bbe298e3d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.406673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c8374-a5e7-420f-b0f9-f2bbe298e3d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.407994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c8374-a5e7-420f-b0f9-f2bbe298e3d8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.554644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 992/2024] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/projects => generated 389 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:12.555753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994617f-85d1-4bca-82f8-ebc2d11bd57f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.566909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994617f-85d1-4bca-82f8-ebc2d11bd57f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.592418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:12.635174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73851a-2d56-47ca-9a69-18c84df87621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.637154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73851a-2d56-47ca-9a69-18c84df87621 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.637680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73851a-2d56-47ca-9a69-18c84df87621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.637961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73851a-2d56-47ca-9a69-18c84df87621 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.638603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73851a-2d56-47ca-9a69-18c84df87621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.641307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73851a-2d56-47ca-9a69-18c84df87621 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.720893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1033/2025] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:12.722088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c8374-a5e7-420f-b0f9-f2bbe298e3d8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.734459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c8374-a5e7-420f-b0f9-f2bbe298e3d8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.739674 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7bade-c668-4f5d-a2ea-503adf315b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.740313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7bade-c668-4f5d-a2ea-503adf315b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.740590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7bade-c668-4f5d-a2ea-503adf315b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.740882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7bade-c668-4f5d-a2ea-503adf315b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.743148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7bade-c668-4f5d-a2ea-503adf315b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.743148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7bade-c668-4f5d-a2ea-503adf315b59 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.753516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 993/2026] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:12.758036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73851a-2d56-47ca-9a69-18c84df87621 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.769755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73851a-2d56-47ca-9a69-18c84df87621 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.777677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa3a46-3c2b-4e03-a99e-41a53ce1e93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.778308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa3a46-3c2b-4e03-a99e-41a53ce1e93e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.778588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa3a46-3c2b-4e03-a99e-41a53ce1e93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.778861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa3a46-3c2b-4e03-a99e-41a53ce1e93e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.779510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa3a46-3c2b-4e03-a99e-41a53ce1e93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.781130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa3a46-3c2b-4e03-a99e-41a53ce1e93e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.796683 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1034/2027] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:12.798316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:12.801402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7bade-c668-4f5d-a2ea-503adf315b59 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.832952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7bade-c668-4f5d-a2ea-503adf315b59 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.840382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7fc60-57c5-499f-89af-bfda7effc972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.841602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7fc60-57c5-499f-89af-bfda7effc972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.841985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7fc60-57c5-499f-89af-bfda7effc972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.842350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7fc60-57c5-499f-89af-bfda7effc972 admin admin] PATH_INFO: `/v3/projects/65d30326a4984579b2c59c26ffd68cd6/users/b645424467b94569ab9fb821acab6935/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.861360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7fc60-57c5-499f-89af-bfda7effc972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d30326a4984579b2c59c26ffd68cd6, user_id=b645424467b94569ab9fb821acab6935, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.864194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7fc60-57c5-499f-89af-bfda7effc972 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.946113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 994/2028] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:12 2022] PUT /identity/v3/projects/65d30326a4984579b2c59c26ffd68cd6/users/b645424467b94569ab9fb821acab6935/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:12.956569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75182eb6-e21b-4e54-83c5-889657636f94 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.957031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75182eb6-e21b-4e54-83c5-889657636f94 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.957121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75182eb6-e21b-4e54-83c5-889657636f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.988661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-75182eb6-e21b-4e54-83c5-889657636f94 None None] MFA Rules not processed for user `b645424467b94569ab9fb821acab6935`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:12.990307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-75182eb6-e21b-4e54-83c5-889657636f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:13.096511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1035/2029] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:13.110285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7fc60-57c5-499f-89af-bfda7effc972 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.181953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7fc60-57c5-499f-89af-bfda7effc972 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.189401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1027a11-0193-4e1a-884c-02ca20500daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.190237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1027a11-0193-4e1a-884c-02ca20500daf admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.190937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1027a11-0193-4e1a-884c-02ca20500daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.191593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1027a11-0193-4e1a-884c-02ca20500daf admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.192711 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1027a11-0193-4e1a-884c-02ca20500daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.196037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1027a11-0193-4e1a-884c-02ca20500daf admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.311780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 995/2030] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:13 2022] POST /identity/v3/projects => generated 389 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:13.312999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75182eb6-e21b-4e54-83c5-889657636f94 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.324627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75182eb6-e21b-4e54-83c5-889657636f94 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.372539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:13.388196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b87ac8-5b4e-4211-82c6-aed6eec2585a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.388798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b87ac8-5b4e-4211-82c6-aed6eec2585a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.389074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b87ac8-5b4e-4211-82c6-aed6eec2585a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.389343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b87ac8-5b4e-4211-82c6-aed6eec2585a admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.389994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b87ac8-5b4e-4211-82c6-aed6eec2585a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.392309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b87ac8-5b4e-4211-82c6-aed6eec2585a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.493782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1036/2031] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:13 2022] POST /identity/v3/users => generated 301 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:13.498709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1027a11-0193-4e1a-884c-02ca20500daf admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.509545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1027a11-0193-4e1a-884c-02ca20500daf admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.515645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:13.516802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6d287-40cd-4c3f-8bc2-923fc6a0d86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.517631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6d287-40cd-4c3f-8bc2-923fc6a0d86e admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.517943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6d287-40cd-4c3f-8bc2-923fc6a0d86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.518464 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6d287-40cd-4c3f-8bc2-923fc6a0d86e admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.519155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6d287-40cd-4c3f-8bc2-923fc6a0d86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.520981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6d287-40cd-4c3f-8bc2-923fc6a0d86e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.533857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 996/2032] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:13.538398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b87ac8-5b4e-4211-82c6-aed6eec2585a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.549119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b87ac8-5b4e-4211-82c6-aed6eec2585a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.554594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c75c8-69a2-42f1-88c7-780ca1fd80c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.555243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c75c8-69a2-42f1-88c7-780ca1fd80c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.560795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c75c8-69a2-42f1-88c7-780ca1fd80c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.560795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c75c8-69a2-42f1-88c7-780ca1fd80c5 admin admin] PATH_INFO: `/v3/projects/d56a30030a8442ee84a912ef64bd3bcb/users/6206739df4e84844bdc37d64587c9f43/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.577726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c75c8-69a2-42f1-88c7-780ca1fd80c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56a30030a8442ee84a912ef64bd3bcb, user_id=6206739df4e84844bdc37d64587c9f43, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.583916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c75c8-69a2-42f1-88c7-780ca1fd80c5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.675097 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:13.681104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1037/2033] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:13 2022] PUT /identity/v3/projects/d56a30030a8442ee84a912ef64bd3bcb/users/6206739df4e84844bdc37d64587c9f43/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:13.687100 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6d287-40cd-4c3f-8bc2-923fc6a0d86e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.697121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6d287-40cd-4c3f-8bc2-923fc6a0d86e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.740106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:13.756558 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19544db-1e92-4fc9-9206-0be2225f3709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.757123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19544db-1e92-4fc9-9206-0be2225f3709 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.757400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19544db-1e92-4fc9-9206-0be2225f3709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.757708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19544db-1e92-4fc9-9206-0be2225f3709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.758348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19544db-1e92-4fc9-9206-0be2225f3709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.759930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19544db-1e92-4fc9-9206-0be2225f3709 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.772716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 997/2034] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:13.774965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c75c8-69a2-42f1-88c7-780ca1fd80c5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.785964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c75c8-69a2-42f1-88c7-780ca1fd80c5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.790505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4bff3-7747-4788-ae6b-06792d2efdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.791145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4bff3-7747-4788-ae6b-06792d2efdbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.791421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4bff3-7747-4788-ae6b-06792d2efdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.791687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4bff3-7747-4788-ae6b-06792d2efdbf admin admin] PATH_INFO: `/v3/projects/d56a30030a8442ee84a912ef64bd3bcb/users/6206739df4e84844bdc37d64587c9f43/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.794024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4bff3-7747-4788-ae6b-06792d2efdbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56a30030a8442ee84a912ef64bd3bcb, user_id=6206739df4e84844bdc37d64587c9f43, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.796815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4bff3-7747-4788-ae6b-06792d2efdbf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.807567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1038/2035] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:13 2022] PUT /identity/v3/projects/d56a30030a8442ee84a912ef64bd3bcb/users/6206739df4e84844bdc37d64587c9f43/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:13.819475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a9dc-fefb-4016-a2ea-c36844603eab None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.819757 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a9dc-fefb-4016-a2ea-c36844603eab None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.820030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a9dc-fefb-4016-a2ea-c36844603eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.849670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5741a9dc-fefb-4016-a2ea-c36844603eab None None] MFA Rules not processed for user `6206739df4e84844bdc37d64587c9f43`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:13.851191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5741a9dc-fefb-4016-a2ea-c36844603eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:13.929396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 998/2036] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:13 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:13.941707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4bff3-7747-4788-ae6b-06792d2efdbf admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.943384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1ac4bff3-7747-4788-ae6b-06792d2efdbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:14.003033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4bff3-7747-4788-ae6b-06792d2efdbf admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:14.006822 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c825e28-66e9-4fe0-a14a-83194b37479b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:14.007426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c825e28-66e9-4fe0-a14a-83194b37479b service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:14.007701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c825e28-66e9-4fe0-a14a-83194b37479b service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:14.007972 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c825e28-66e9-4fe0-a14a-83194b37479b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:14.018050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c825e28-66e9-4fe0-a14a-83194b37479b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:14.019469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c825e28-66e9-4fe0-a14a-83194b37479b service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:14.030219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1039/2037] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:13 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:14.037568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:14.173417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741a9dc-fefb-4016-a2ea-c36844603eab None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:14.238222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741a9dc-fefb-4016-a2ea-c36844603eab None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:14.243033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:14.243697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:14.243947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:14.244218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] PATH_INFO: `/v3/users/b645424467b94569ab9fb821acab6935` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:14.245382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b645424467b94569ab9fb821acab6935)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:14.247514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:14.261883 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:14.450083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b645424467b94569ab9fb821acab6935'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:14.450813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b645424467b94569ab9fb821acab6935'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:14.520282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b645424467b94569ab9fb821acab6935'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:14.534970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b645424467b94569ab9fb821acab6935'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:14.543064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b645424467b94569ab9fb821acab6935'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:14.549421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 999/2038] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:14 2022] DELETE /identity/v3/users/b645424467b94569ab9fb821acab6935 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:14.554452 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce21f1e-36f5-4597-a740-68fc465576c0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:14.554770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce21f1e-36f5-4597-a740-68fc465576c0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:14.555053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce21f1e-36f5-4597-a740-68fc465576c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:14.577675 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8ce21f1e-36f5-4597-a740-68fc465576c0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:14.578799 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8ce21f1e-36f5-4597-a740-68fc465576c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:14.639607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1040/2039] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:14.649788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:14.651195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:14.718453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f9ff2-39ae-4961-8ced-634541409770 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:14.722614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47e3b-c483-47be-98a9-1a53b8744565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:14.723499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47e3b-c483-47be-98a9-1a53b8744565 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:14.723790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47e3b-c483-47be-98a9-1a53b8744565 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:14.729684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be47e3b-c483-47be-98a9-1a53b8744565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:14.739030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be47e3b-c483-47be-98a9-1a53b8744565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:14.740510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be47e3b-c483-47be-98a9-1a53b8744565 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:14.750984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1000/2040] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:15.012037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce21f1e-36f5-4597-a740-68fc465576c0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:15.013508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8ce21f1e-36f5-4597-a740-68fc465576c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:15.027119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce21f1e-36f5-4597-a740-68fc465576c0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:15.032248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:15.032890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:15.033170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:15.033486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] PATH_INFO: `/v3/users/a9cf28e775ff4f6fa4a107522a53c184` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:15.034765 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9cf28e775ff4f6fa4a107522a53c184)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:15.036713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:15.192949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47e3b-c483-47be-98a9-1a53b8744565 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:15.194892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3be47e3b-c483-47be-98a9-1a53b8744565 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:15.211219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be47e3b-c483-47be-98a9-1a53b8744565 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:15.226021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:15.226704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:15.227079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:15.227320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] PATH_INFO: `/v3/projects/65d30326a4984579b2c59c26ffd68cd6` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:15.228466 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d30326a4984579b2c59c26ffd68cd6)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:15.232062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:15.236948 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9cf28e775ff4f6fa4a107522a53c184'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.249155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9cf28e775ff4f6fa4a107522a53c184'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.256206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9cf28e775ff4f6fa4a107522a53c184'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.320475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9cf28e775ff4f6fa4a107522a53c184'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.321116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e2920328-e494-4499-86b8-d990a1833d66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9cf28e775ff4f6fa4a107522a53c184'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.328407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1041/2041] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:15 2022] DELETE /identity/v3/users/a9cf28e775ff4f6fa4a107522a53c184 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:15.332945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773380-993e-4edf-9774-b568cb1529e8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:15.333236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773380-993e-4edf-9774-b568cb1529e8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:15.333554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773380-993e-4edf-9774-b568cb1529e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:15.352943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ad773380-993e-4edf-9774-b568cb1529e8 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:15.354120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ad773380-993e-4edf-9774-b568cb1529e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:15.370726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d30326a4984579b2c59c26ffd68cd6', 'request_id': 'req-ee9ab057-2239-451f-b7a2-2675667fd049'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.408333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:15.418474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1042/2042] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:15.428727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad773380-993e-4edf-9774-b568cb1529e8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:15.430148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ad773380-993e-4edf-9774-b568cb1529e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:15.434898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d30326a4984579b2c59c26ffd68cd6', 'request_id': 'req-ee9ab057-2239-451f-b7a2-2675667fd049'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.442362 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d30326a4984579b2c59c26ffd68cd6', 'request_id': 'req-ee9ab057-2239-451f-b7a2-2675667fd049'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.455754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] The token cache is being invalidate because project 65d30326a4984579b2c59c26ffd68cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:15.456216 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.457004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.461563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1001/2043] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:15 2022] DELETE /identity/v3/projects/65d30326a4984579b2c59c26ffd68cd6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:15.462526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:15.464039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:15.478968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ab057-2239-451f-b7a2-2675667fd049 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:15.484721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:15.485483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:15.485776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:15.486741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] PATH_INFO: `/v3/users/6206739df4e84844bdc37d64587c9f43` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:15.487210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6206739df4e84844bdc37d64587c9f43)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:15.489255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:15.498667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad773380-993e-4edf-9774-b568cb1529e8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:15.502831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:15.503724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34c61f-e92e-472a-b21c-9f71391740fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:15.504447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34c61f-e92e-472a-b21c-9f71391740fe service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:15.505709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34c61f-e92e-472a-b21c-9f71391740fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:15.505998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34c61f-e92e-472a-b21c-9f71391740fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:15.510296 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ab34c61f-e92e-472a-b21c-9f71391740fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:15.525782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34c61f-e92e-472a-b21c-9f71391740fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:15.527312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34c61f-e92e-472a-b21c-9f71391740fe service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:15.541726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1043/2044] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:32:15.637776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6206739df4e84844bdc37d64587c9f43'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.638585 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6206739df4e84844bdc37d64587c9f43'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.694611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6206739df4e84844bdc37d64587c9f43'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.709023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6206739df4e84844bdc37d64587c9f43'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.726582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6206739df4e84844bdc37d64587c9f43'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:15.734529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1002/2045] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:15 2022] DELETE /identity/v3/users/6206739df4e84844bdc37d64587c9f43 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:15.936725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34c61f-e92e-472a-b21c-9f71391740fe service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:15.938029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ab34c61f-e92e-472a-b21c-9f71391740fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:15.957959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34c61f-e92e-472a-b21c-9f71391740fe service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:16.026720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:16.027492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:16.027829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:16.028108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] PATH_INFO: `/v3/projects/5716551007a448408d043f7d84edb1b0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:16.029258 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] RBAC: Authorizing `identity:delete_project(project_id=5716551007a448408d043f7d84edb1b0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:16.031207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:16.137784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5716551007a448408d043f7d84edb1b0', 'request_id': 'req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.182186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:16.185252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:16.212413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceea4e7-148c-439b-a245-fbf533b8e12e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:16.214619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5716551007a448408d043f7d84edb1b0', 'request_id': 'req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.225297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5716551007a448408d043f7d84edb1b0', 'request_id': 'req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.236232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] The token cache is being invalidate because project 5716551007a448408d043f7d84edb1b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:16.236518 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.237156 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.238678 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1044/2046] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:15 2022] DELETE /identity/v3/projects/5716551007a448408d043f7d84edb1b0 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:16.242582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:16.243947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:16.285168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:16.289392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:16.290331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:16.290587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:16.290961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] PATH_INFO: `/v3/projects/d56a30030a8442ee84a912ef64bd3bcb` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:16.292832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56a30030a8442ee84a912ef64bd3bcb)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:16.294987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:16.320052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ad28f-1714-4f2e-a0a2-77bb5e8d65af admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:16.325559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:16.326194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:16.326508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:16.326810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] PATH_INFO: `/v3/users/37dcef4a31af44e9b8dd0f6d7e5a4c44` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:16.329503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=37dcef4a31af44e9b8dd0f6d7e5a4c44)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:16.330105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:16.428215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56a30030a8442ee84a912ef64bd3bcb', 'request_id': 'req-77e97b9e-37fb-4c33-92a4-b36cb1239d22'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.499585 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56a30030a8442ee84a912ef64bd3bcb', 'request_id': 'req-77e97b9e-37fb-4c33-92a4-b36cb1239d22'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.506678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56a30030a8442ee84a912ef64bd3bcb', 'request_id': 'req-77e97b9e-37fb-4c33-92a4-b36cb1239d22'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.517649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] The token cache is being invalidate because project d56a30030a8442ee84a912ef64bd3bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:16.517937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.518853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-77e97b9e-37fb-4c33-92a4-b36cb1239d22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.520865 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1003/2047] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:16 2022] DELETE /identity/v3/projects/d56a30030a8442ee84a912ef64bd3bcb => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:16.525544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb2333-6aa1-4766-b5c4-3d941a07b32e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:16.525828 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb2333-6aa1-4766-b5c4-3d941a07b32e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:16.526102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb2333-6aa1-4766-b5c4-3d941a07b32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:16.545779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37dcef4a31af44e9b8dd0f6d7e5a4c44'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.553366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-decb2333-6aa1-4766-b5c4-3d941a07b32e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:16.554628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-decb2333-6aa1-4766-b5c4-3d941a07b32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:16.559208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37dcef4a31af44e9b8dd0f6d7e5a4c44'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.566365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37dcef4a31af44e9b8dd0f6d7e5a4c44'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.567939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:16.625346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1004/2048] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:16.632940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc369a9-2846-4109-a047-f633283cb4cc None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:16.633304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc369a9-2846-4109-a047-f633283cb4cc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:16.633609 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc369a9-2846-4109-a047-f633283cb4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:16.637055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37dcef4a31af44e9b8dd0f6d7e5a4c44'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.637827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37dcef4a31af44e9b8dd0f6d7e5a4c44'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:16.644993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1045/2049] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:16 2022] DELETE /identity/v3/users/37dcef4a31af44e9b8dd0f6d7e5a4c44 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:16.656506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5bc369a9-2846-4109-a047-f633283cb4cc None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:16.658824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5bc369a9-2846-4109-a047-f633283cb4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:16.749504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1005/2050] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:16.751094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:16.761939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48afc04-9de5-4291-a687-1066a694a88f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:16.766628 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2d3d7-4926-4951-a349-18d05196dd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:16.767428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2d3d7-4926-4951-a349-18d05196dd36 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:16.767773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2d3d7-4926-4951-a349-18d05196dd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:16.768118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2d3d7-4926-4951-a349-18d05196dd36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:16.768861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2d3d7-4926-4951-a349-18d05196dd36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:16.769228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2d3d7-4926-4951-a349-18d05196dd36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:16.771082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2d3d7-4926-4951-a349-18d05196dd36 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:16.785290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1046/2051] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:32:16.789413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc369a9-2846-4109-a047-f633283cb4cc None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:16.799645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc369a9-2846-4109-a047-f633283cb4cc None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:16.804428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd627b14-eb2e-492b-ab77-a8291d94317c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:16.805051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd627b14-eb2e-492b-ab77-a8291d94317c admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:16.805330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd627b14-eb2e-492b-ab77-a8291d94317c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:16.805636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd627b14-eb2e-492b-ab77-a8291d94317c admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:16.806343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd627b14-eb2e-492b-ab77-a8291d94317c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:16.808995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd627b14-eb2e-492b-ab77-a8291d94317c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:16.966237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1006/2052] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:16 2022] POST /identity/v3/projects => generated 409 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:16.967704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2d3d7-4926-4951-a349-18d05196dd36 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:16.978371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2d3d7-4926-4951-a349-18d05196dd36 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.039529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456976f-27a9-436d-93bd-624d8bd9a9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.040136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456976f-27a9-436d-93bd-624d8bd9a9b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.040408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456976f-27a9-436d-93bd-624d8bd9a9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.040713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456976f-27a9-436d-93bd-624d8bd9a9b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.041340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456976f-27a9-436d-93bd-624d8bd9a9b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.043315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456976f-27a9-436d-93bd-624d8bd9a9b0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.125486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd627b14-eb2e-492b-ab77-a8291d94317c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.127227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fd627b14-eb2e-492b-ab77-a8291d94317c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:17.141334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd627b14-eb2e-492b-ab77-a8291d94317c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.148472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.149639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.149639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.149939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] PATH_INFO: `/v3/projects/4577656f83bd4224952086482857116a` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.152822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=4577656f83bd4224952086482857116a)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.158343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.161423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1047/2053] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:16 2022] POST /identity/v3/users => generated 305 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:17.166896 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456976f-27a9-436d-93bd-624d8bd9a9b0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.177041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456976f-27a9-436d-93bd-624d8bd9a9b0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.181667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fec15-b509-4a88-a3ae-0d8e8fc725c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.182255 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fec15-b509-4a88-a3ae-0d8e8fc725c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.182529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fec15-b509-4a88-a3ae-0d8e8fc725c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.183232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fec15-b509-4a88-a3ae-0d8e8fc725c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.184059 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4fec15-b509-4a88-a3ae-0d8e8fc725c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.185689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4fec15-b509-4a88-a3ae-0d8e8fc725c8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.196494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1048/2054] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:32:17.200990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fec15-b509-4a88-a3ae-0d8e8fc725c8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.210685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fec15-b509-4a88-a3ae-0d8e8fc725c8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.221596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e86ff-2c89-4c41-87b0-401b47b4b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.221596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e86ff-2c89-4c41-87b0-401b47b4b62d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.221596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e86ff-2c89-4c41-87b0-401b47b4b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.221596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e86ff-2c89-4c41-87b0-401b47b4b62d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.222329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e86ff-2c89-4c41-87b0-401b47b4b62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.222329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e86ff-2c89-4c41-87b0-401b47b4b62d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.231826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1049/2055] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:32:17.236054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e86ff-2c89-4c41-87b0-401b47b4b62d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.242401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:17.250399 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e86ff-2c89-4c41-87b0-401b47b4b62d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.252843 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d1bd9-1cc9-436c-9d31-19a2b28b9593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.253525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d1bd9-1cc9-436c-9d31-19a2b28b9593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.253877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d1bd9-1cc9-436c-9d31-19a2b28b9593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.254149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d1bd9-1cc9-436c-9d31-19a2b28b9593 admin admin] PATH_INFO: `/v3/projects/41907acf53304acd97a3ea430b8a5b59/users/31d2677b05e74b5c9bd53324cbe7d289/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.275518 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d1bd9-1cc9-436c-9d31-19a2b28b9593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41907acf53304acd97a3ea430b8a5b59, user_id=31d2677b05e74b5c9bd53324cbe7d289, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.278946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d1bd9-1cc9-436c-9d31-19a2b28b9593 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.287047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4577656f83bd4224952086482857116a', 'request_id': 'req-6a43ed44-75fa-495f-94a1-d038ece513ee'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:17.357484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1050/2056] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:17 2022] PUT /identity/v3/projects/41907acf53304acd97a3ea430b8a5b59/users/31d2677b05e74b5c9bd53324cbe7d289/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:17.357683 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4577656f83bd4224952086482857116a', 'request_id': 'req-6a43ed44-75fa-495f-94a1-d038ece513ee'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:17.358527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4577656f83bd4224952086482857116a', 'request_id': 'req-6a43ed44-75fa-495f-94a1-d038ece513ee'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:17.369042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d98a9-766c-4f76-9e3e-e11723662cb5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.369322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d98a9-766c-4f76-9e3e-e11723662cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.369424 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] The token cache is being invalidate because project 4577656f83bd4224952086482857116a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:17.369640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d98a9-766c-4f76-9e3e-e11723662cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.369740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:17.371031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6a43ed44-75fa-495f-94a1-d038ece513ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:17.379569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1007/2057] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:17 2022] DELETE /identity/v3/projects/4577656f83bd4224952086482857116a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:17.382999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a08524-dfa5-44cc-adb6-31a99638bc53 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.383293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a08524-dfa5-44cc-adb6-31a99638bc53 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.383567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a08524-dfa5-44cc-adb6-31a99638bc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.398789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-349d98a9-766c-4f76-9e3e-e11723662cb5 None None] MFA Rules not processed for user `31d2677b05e74b5c9bd53324cbe7d289`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:17.400011 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-349d98a9-766c-4f76-9e3e-e11723662cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:17.404945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-61a08524-dfa5-44cc-adb6-31a99638bc53 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:17.406179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-61a08524-dfa5-44cc-adb6-31a99638bc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:17.420772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:17.467356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1051/2058] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:17 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:17.469154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1008/2059] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:17.471902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d98a9-766c-4f76-9e3e-e11723662cb5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.473145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-349d98a9-766c-4f76-9e3e-e11723662cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:17.476733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ab001-ea7a-4eb1-b2e5-16cc1b1e332e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.477021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ab001-ea7a-4eb1-b2e5-16cc1b1e332e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.477413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ab001-ea7a-4eb1-b2e5-16cc1b1e332e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.489668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d98a9-766c-4f76-9e3e-e11723662cb5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.494520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cc29b-0ed9-4c6d-a1b0-227e0c6830a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.495694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cc29b-0ed9-4c6d-a1b0-227e0c6830a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.496016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cc29b-0ed9-4c6d-a1b0-227e0c6830a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.496290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cc29b-0ed9-4c6d-a1b0-227e0c6830a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.496929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cc29b-0ed9-4c6d-a1b0-227e0c6830a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.498899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cc29b-0ed9-4c6d-a1b0-227e0c6830a5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.500642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a09ab001-ea7a-4eb1-b2e5-16cc1b1e332e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:17.501913 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a09ab001-ea7a-4eb1-b2e5-16cc1b1e332e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:17.511345 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1009/2060] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:17.515816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ab001-ea7a-4eb1-b2e5-16cc1b1e332e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.524889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ab001-ea7a-4eb1-b2e5-16cc1b1e332e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.530409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aef063-3f93-41c5-9d98-6cd1f75c785c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.530996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aef063-3f93-41c5-9d98-6cd1f75c785c admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.531295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aef063-3f93-41c5-9d98-6cd1f75c785c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.531564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aef063-3f93-41c5-9d98-6cd1f75c785c admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.532228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aef063-3f93-41c5-9d98-6cd1f75c785c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:17.532548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aef063-3f93-41c5-9d98-6cd1f75c785c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.534186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aef063-3f93-41c5-9d98-6cd1f75c785c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.547639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1010/2061] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:32:17.551806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aef063-3f93-41c5-9d98-6cd1f75c785c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.561941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aef063-3f93-41c5-9d98-6cd1f75c785c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.566722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5eea00-ff8a-4110-999e-309d705b4796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.567305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5eea00-ff8a-4110-999e-309d705b4796 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.567586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5eea00-ff8a-4110-999e-309d705b4796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.567859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5eea00-ff8a-4110-999e-309d705b4796 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.568481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5eea00-ff8a-4110-999e-309d705b4796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.570397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5eea00-ff8a-4110-999e-309d705b4796 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.584299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1052/2062] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:17 2022] POST /identity/v3/projects => generated 407 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:17.588323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cc29b-0ed9-4c6d-a1b0-227e0c6830a5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.652222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cc29b-0ed9-4c6d-a1b0-227e0c6830a5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.656944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1da4-c592-4901-b75a-59bd2c0157d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.657571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1da4-c592-4901-b75a-59bd2c0157d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.657852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1da4-c592-4901-b75a-59bd2c0157d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.658142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1da4-c592-4901-b75a-59bd2c0157d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.658759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf1da4-c592-4901-b75a-59bd2c0157d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.660680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf1da4-c592-4901-b75a-59bd2c0157d8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.670869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1011/2063] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:17 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:17.675656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5eea00-ff8a-4110-999e-309d705b4796 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.738458 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5eea00-ff8a-4110-999e-309d705b4796 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.746589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c131f-3994-451f-9126-e8e3e39a616b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.747272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c131f-3994-451f-9126-e8e3e39a616b admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.747551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c131f-3994-451f-9126-e8e3e39a616b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.747821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c131f-3994-451f-9126-e8e3e39a616b admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.748459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c131f-3994-451f-9126-e8e3e39a616b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.753359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1053/2064] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:17 2022] POST /identity/v3/users => generated 310 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:17.753537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c131f-3994-451f-9126-e8e3e39a616b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.754522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1da4-c592-4901-b75a-59bd2c0157d8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.765332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1da4-c592-4901-b75a-59bd2c0157d8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.770018 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7298488-87cb-40ee-bd2b-3d42c0aa3ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.770620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7298488-87cb-40ee-bd2b-3d42c0aa3ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.770899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7298488-87cb-40ee-bd2b-3d42c0aa3ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.771169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7298488-87cb-40ee-bd2b-3d42c0aa3ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.771807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7298488-87cb-40ee-bd2b-3d42c0aa3ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.773426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7298488-87cb-40ee-bd2b-3d42c0aa3ec8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.783384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1054/2065] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:17.787384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7298488-87cb-40ee-bd2b-3d42c0aa3ec8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.797189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7298488-87cb-40ee-bd2b-3d42c0aa3ec8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.802254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7baa44-2187-4a99-9024-898553dd230f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.802959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7baa44-2187-4a99-9024-898553dd230f admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.803058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7baa44-2187-4a99-9024-898553dd230f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.803386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7baa44-2187-4a99-9024-898553dd230f admin admin] PATH_INFO: `/v3/projects/450e7628dea04032b32070b0a666033b/users/be1403f5de5f400f91cf0ccb441fa8ee/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.821805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7baa44-2187-4a99-9024-898553dd230f admin admin] RBAC: Authorizing `identity:create_grant(project_id=450e7628dea04032b32070b0a666033b, user_id=be1403f5de5f400f91cf0ccb441fa8ee, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.824910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7baa44-2187-4a99-9024-898553dd230f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.826288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1012/2066] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:17 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:17.830280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c131f-3994-451f-9126-e8e3e39a616b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.841447 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c131f-3994-451f-9126-e8e3e39a616b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.846492 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a1624-fea9-4562-9858-4c2359f443d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.847091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a1624-fea9-4562-9858-4c2359f443d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.847390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a1624-fea9-4562-9858-4c2359f443d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.852265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a1624-fea9-4562-9858-4c2359f443d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.852265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a1624-fea9-4562-9858-4c2359f443d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.852265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a1624-fea9-4562-9858-4c2359f443d2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.863303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1013/2067] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:32:17.867185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a1624-fea9-4562-9858-4c2359f443d2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.878155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a1624-fea9-4562-9858-4c2359f443d2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.882982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe5e66-9696-417c-ba42-c5acae1c2433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.883578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe5e66-9696-417c-ba42-c5acae1c2433 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.883859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe5e66-9696-417c-ba42-c5acae1c2433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.884135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe5e66-9696-417c-ba42-c5acae1c2433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.884771 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe5e66-9696-417c-ba42-c5acae1c2433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.886986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe5e66-9696-417c-ba42-c5acae1c2433 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:17.900921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1014/2068] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:32:17.903999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe5e66-9696-417c-ba42-c5acae1c2433 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.909491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1055/2069] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:17 2022] PUT /identity/v3/projects/450e7628dea04032b32070b0a666033b/users/be1403f5de5f400f91cf0ccb441fa8ee/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:17.912984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7baa44-2187-4a99-9024-898553dd230f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:17.918159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:17.921646 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe5e66-9696-417c-ba42-c5acae1c2433 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.974723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:17.990920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7baa44-2187-4a99-9024-898553dd230f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:17.993660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabac8e-3fac-44a4-bd5a-a35c218fa34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.994341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabac8e-3fac-44a4-bd5a-a35c218fa34c admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.994617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabac8e-3fac-44a4-bd5a-a35c218fa34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.994922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabac8e-3fac-44a4-bd5a-a35c218fa34c admin admin] PATH_INFO: `/v3/projects/6f901db083e7435ba2f1c2b03febeb1f/users/19fd2c60d6c24b18b3b93b4b67729589/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.996110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373647f1-5334-457f-b531-91b835d2648f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:17.996724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373647f1-5334-457f-b531-91b835d2648f admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:17.997008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373647f1-5334-457f-b531-91b835d2648f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:17.997288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373647f1-5334-457f-b531-91b835d2648f admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:17.997977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373647f1-5334-457f-b531-91b835d2648f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:17.999613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373647f1-5334-457f-b531-91b835d2648f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:18.009792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1056/2070] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:18.014694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373647f1-5334-457f-b531-91b835d2648f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:18.016444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabac8e-3fac-44a4-bd5a-a35c218fa34c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f901db083e7435ba2f1c2b03febeb1f, user_id=19fd2c60d6c24b18b3b93b4b67729589, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:18.019204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabac8e-3fac-44a4-bd5a-a35c218fa34c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:18.024808 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373647f1-5334-457f-b531-91b835d2648f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:18.030542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510fc50-9850-4da7-91de-3a228220cb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:18.031294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510fc50-9850-4da7-91de-3a228220cb1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:18.031637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510fc50-9850-4da7-91de-3a228220cb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:18.031976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510fc50-9850-4da7-91de-3a228220cb1f admin admin] PATH_INFO: `/v3/projects/450e7628dea04032b32070b0a666033b/users/be1403f5de5f400f91cf0ccb441fa8ee/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:18.033946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510fc50-9850-4da7-91de-3a228220cb1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=450e7628dea04032b32070b0a666033b, user_id=be1403f5de5f400f91cf0ccb441fa8ee, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:18.036593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510fc50-9850-4da7-91de-3a228220cb1f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:18.047320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1057/2071] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:18 2022] PUT /identity/v3/projects/450e7628dea04032b32070b0a666033b/users/be1403f5de5f400f91cf0ccb441fa8ee/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:18.057404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c325df-09aa-4c21-9707-9b95020505aa None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:18.058259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c325df-09aa-4c21-9707-9b95020505aa None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:18.058568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c325df-09aa-4c21-9707-9b95020505aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:18.071249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1015/2072] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:17 2022] PUT /identity/v3/projects/6f901db083e7435ba2f1c2b03febeb1f/users/19fd2c60d6c24b18b3b93b4b67729589/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:18.076855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd49239-74fe-415b-82f3-d8c7e68babec None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:18.077235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd49239-74fe-415b-82f3-d8c7e68babec None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:18.077712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd49239-74fe-415b-82f3-d8c7e68babec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:18.095505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a1c325df-09aa-4c21-9707-9b95020505aa None None] MFA Rules not processed for user `be1403f5de5f400f91cf0ccb441fa8ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:18.097128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a1c325df-09aa-4c21-9707-9b95020505aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:18.101740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cdd49239-74fe-415b-82f3-d8c7e68babec None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:18.103284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cdd49239-74fe-415b-82f3-d8c7e68babec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:18.186185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1016/2073] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:18.193495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1058/2074] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:18 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:18.196064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd49239-74fe-415b-82f3-d8c7e68babec None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:18.197888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cdd49239-74fe-415b-82f3-d8c7e68babec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:18.205508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c325df-09aa-4c21-9707-9b95020505aa None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:18.206993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a1c325df-09aa-4c21-9707-9b95020505aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:18.262208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd49239-74fe-415b-82f3-d8c7e68babec None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:18.267021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd16c0a-490f-4ec1-93f0-8fa35bcbbdee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:18.267634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16c0a-490f-4ec1-93f0-8fa35bcbbdee service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:18.267909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16c0a-490f-4ec1-93f0-8fa35bcbbdee service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:18.268181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16c0a-490f-4ec1-93f0-8fa35bcbbdee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:18.269165 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c325df-09aa-4c21-9707-9b95020505aa None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:18.273469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5be3e4-a4ed-41a5-9a1e-e1ec8b6437f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:18.274447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5be3e4-a4ed-41a5-9a1e-e1ec8b6437f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:18.274781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5be3e4-a4ed-41a5-9a1e-e1ec8b6437f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:18.275242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5be3e4-a4ed-41a5-9a1e-e1ec8b6437f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:18.280168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd16c0a-490f-4ec1-93f0-8fa35bcbbdee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:18.281701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd16c0a-490f-4ec1-93f0-8fa35bcbbdee service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:18.289185 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5be3e4-a4ed-41a5-9a1e-e1ec8b6437f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:18.290739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5be3e4-a4ed-41a5-9a1e-e1ec8b6437f7 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:18.292597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1017/2075] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:18.301906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1059/2076] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:18 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:32:18.752468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd16c0a-490f-4ec1-93f0-8fa35bcbbdee service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:18.753888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cfd16c0a-490f-4ec1-93f0-8fa35bcbbdee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:18.766638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd16c0a-490f-4ec1-93f0-8fa35bcbbdee service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:18.770244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc049590-aa48-47d1-8fab-107234d58aba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:18.770869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc049590-aa48-47d1-8fab-107234d58aba service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:18.771155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc049590-aa48-47d1-8fab-107234d58aba service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:18.771430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc049590-aa48-47d1-8fab-107234d58aba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:18.832578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc049590-aa48-47d1-8fab-107234d58aba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:18.834086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc049590-aa48-47d1-8fab-107234d58aba service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:18.848496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1018/2077] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:18 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:19.748405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff431a9-842f-42d3-8654-a4f025d63f33 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:19.748956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff431a9-842f-42d3-8654-a4f025d63f33 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:19.749269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff431a9-842f-42d3-8654-a4f025d63f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:19.787825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:19.791170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7ff431a9-842f-42d3-8654-a4f025d63f33 None None] MFA Rules not processed for user `19fd2c60d6c24b18b3b93b4b67729589`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:19.797042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7ff431a9-842f-42d3-8654-a4f025d63f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:19.872930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1060/2078] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:19.877519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc049590-aa48-47d1-8fab-107234d58aba service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:19.889310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc049590-aa48-47d1-8fab-107234d58aba service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:19.893877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d01541-fe2d-4dfc-a440-6a434e4724c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:19.894496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d01541-fe2d-4dfc-a440-6a434e4724c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:19.894775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d01541-fe2d-4dfc-a440-6a434e4724c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:19.895080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d01541-fe2d-4dfc-a440-6a434e4724c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:19.895717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d01541-fe2d-4dfc-a440-6a434e4724c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:19.897670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d01541-fe2d-4dfc-a440-6a434e4724c1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:19.907620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:20.024589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1019/2079] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:19 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:20.028494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff431a9-842f-42d3-8654-a4f025d63f33 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:20.044110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff431a9-842f-42d3-8654-a4f025d63f33 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:20.104233 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51d7f5-ba84-4f0f-a7a5-d5a175003f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:20.104908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51d7f5-ba84-4f0f-a7a5-d5a175003f98 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.105192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51d7f5-ba84-4f0f-a7a5-d5a175003f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.105522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51d7f5-ba84-4f0f-a7a5-d5a175003f98 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.106152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51d7f5-ba84-4f0f-a7a5-d5a175003f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:20.108076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51d7f5-ba84-4f0f-a7a5-d5a175003f98 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:20.215334 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1061/2080] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:20 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:20.219521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d01541-fe2d-4dfc-a440-6a434e4724c1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:20.230253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d01541-fe2d-4dfc-a440-6a434e4724c1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:20.234886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b52ec-6122-4d9c-8209-cb190e7961bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:20.235526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b52ec-6122-4d9c-8209-cb190e7961bf admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.235805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b52ec-6122-4d9c-8209-cb190e7961bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.236072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b52ec-6122-4d9c-8209-cb190e7961bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.236814 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b52ec-6122-4d9c-8209-cb190e7961bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:20.238341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b52ec-6122-4d9c-8209-cb190e7961bf admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:20.254087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1020/2081] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:20.255680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51d7f5-ba84-4f0f-a7a5-d5a175003f98 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:20.267422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51d7f5-ba84-4f0f-a7a5-d5a175003f98 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:20.273275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b17be-adb5-4a62-930f-dd61445d352b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:20.274072 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b17be-adb5-4a62-930f-dd61445d352b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.274350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b17be-adb5-4a62-930f-dd61445d352b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.274620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b17be-adb5-4a62-930f-dd61445d352b admin admin] PATH_INFO: `/v3/projects/5aee6445ce014335a4073ddb51dd1382/users/9d4dbadb297440689aca26af6aab2344/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.292251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b17be-adb5-4a62-930f-dd61445d352b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aee6445ce014335a4073ddb51dd1382, user_id=9d4dbadb297440689aca26af6aab2344, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:20.295110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b17be-adb5-4a62-930f-dd61445d352b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:20.369352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1062/2082] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:20 2022] PUT /identity/v3/projects/5aee6445ce014335a4073ddb51dd1382/users/9d4dbadb297440689aca26af6aab2344/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:20.373934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b52ec-6122-4d9c-8209-cb190e7961bf admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:20.385316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b52ec-6122-4d9c-8209-cb190e7961bf admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:20.451948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d554a0-b7e4-4b38-a8cb-e4e2b86953c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:20.452559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d554a0-b7e4-4b38-a8cb-e4e2b86953c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.452840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d554a0-b7e4-4b38-a8cb-e4e2b86953c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.453114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d554a0-b7e4-4b38-a8cb-e4e2b86953c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.453829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d554a0-b7e4-4b38-a8cb-e4e2b86953c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:20.455529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d554a0-b7e4-4b38-a8cb-e4e2b86953c1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:20.470529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1021/2083] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:20.474501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b17be-adb5-4a62-930f-dd61445d352b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:20.486509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b17be-adb5-4a62-930f-dd61445d352b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:20.491620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76d436-6294-4fff-a55a-2f0c8d038550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:20.492268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d436-6294-4fff-a55a-2f0c8d038550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.492542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d436-6294-4fff-a55a-2f0c8d038550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.492839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d436-6294-4fff-a55a-2f0c8d038550 admin admin] PATH_INFO: `/v3/projects/5aee6445ce014335a4073ddb51dd1382/users/9d4dbadb297440689aca26af6aab2344/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.495087 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76d436-6294-4fff-a55a-2f0c8d038550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aee6445ce014335a4073ddb51dd1382, user_id=9d4dbadb297440689aca26af6aab2344, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:20.498171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76d436-6294-4fff-a55a-2f0c8d038550 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:20.511383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1063/2084] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:20 2022] PUT /identity/v3/projects/5aee6445ce014335a4073ddb51dd1382/users/9d4dbadb297440689aca26af6aab2344/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:21.861394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bdda3-0ba4-4a95-bf8f-bb83a6661e93 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:21.861901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bdda3-0ba4-4a95-bf8f-bb83a6661e93 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:21.862353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bdda3-0ba4-4a95-bf8f-bb83a6661e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:21.901558 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e67bdda3-0ba4-4a95-bf8f-bb83a6661e93 None None] MFA Rules not processed for user `9d4dbadb297440689aca26af6aab2344`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:21.902930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e67bdda3-0ba4-4a95-bf8f-bb83a6661e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:21.986338 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1022/2085] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:21.997994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76d436-6294-4fff-a55a-2f0c8d038550 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:22.055462 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76d436-6294-4fff-a55a-2f0c8d038550 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:22.059976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e070f3d-611c-4aac-9a64-a608915b185a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:22.060559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e070f3d-611c-4aac-9a64-a608915b185a service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:22.060832 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e070f3d-611c-4aac-9a64-a608915b185a service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:22.061099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e070f3d-611c-4aac-9a64-a608915b185a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:22.069311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:22.120183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e070f3d-611c-4aac-9a64-a608915b185a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:22.122032 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e070f3d-611c-4aac-9a64-a608915b185a service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:22.132058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1064/2086] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:31.985016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=72,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:32.224741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=23 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:38.596526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bdda3-0ba4-4a95-bf8f-bb83a6661e93 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:38.607591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bdda3-0ba4-4a95-bf8f-bb83a6661e93 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:38.611373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28e83e-8838-4b04-80ce-380a5f528dcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:38.612234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28e83e-8838-4b04-80ce-380a5f528dcb service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:38.612517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28e83e-8838-4b04-80ce-380a5f528dcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:38.612795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28e83e-8838-4b04-80ce-380a5f528dcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:38.622996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28e83e-8838-4b04-80ce-380a5f528dcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:38.624426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28e83e-8838-4b04-80ce-380a5f528dcb service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:38.636936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1023/2087] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:40.225024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e070f3d-611c-4aac-9a64-a608915b185a service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:40.303935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e070f3d-611c-4aac-9a64-a608915b185a service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:40.310675 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:40.311429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:40.312425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:40.312775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] PATH_INFO: `/v3/users/31d2677b05e74b5c9bd53324cbe7d289` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:40.314053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=31d2677b05e74b5c9bd53324cbe7d289)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:40.316687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:40.548577 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31d2677b05e74b5c9bd53324cbe7d289'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:40.562625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31d2677b05e74b5c9bd53324cbe7d289'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:40.572575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31d2677b05e74b5c9bd53324cbe7d289'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:40.652523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31d2677b05e74b5c9bd53324cbe7d289'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:40.653514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31d2677b05e74b5c9bd53324cbe7d289'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:40.660209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1065/2088] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:40 2022] DELETE /identity/v3/users/31d2677b05e74b5c9bd53324cbe7d289 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:40.664778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e181ef0-3b65-4295-98a5-33ccf69c1127 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:40.665258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e181ef0-3b65-4295-98a5-33ccf69c1127 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:40.665441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e181ef0-3b65-4295-98a5-33ccf69c1127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:40.685265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3e181ef0-3b65-4295-98a5-33ccf69c1127 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:40.686495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3e181ef0-3b65-4295-98a5-33ccf69c1127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:40.750807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1024/2089] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:40.761118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:40.763441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:40.829436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab18577-abc9-4b84-9dfb-8e32bbe25d1b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:40.833240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a07668-6362-415c-9399-2b195644ffe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:40.833911 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a07668-6362-415c-9399-2b195644ffe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:40.834200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a07668-6362-415c-9399-2b195644ffe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:40.834497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a07668-6362-415c-9399-2b195644ffe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:40.844023 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a07668-6362-415c-9399-2b195644ffe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:40.845500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a07668-6362-415c-9399-2b195644ffe9 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:40.854919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1066/2090] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:41.300119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e181ef0-3b65-4295-98a5-33ccf69c1127 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:41.304247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3e181ef0-3b65-4295-98a5-33ccf69c1127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:41.318284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e181ef0-3b65-4295-98a5-33ccf69c1127 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:41.323442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:41.324073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:41.324349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:41.324741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] PATH_INFO: `/v3/projects/41907acf53304acd97a3ea430b8a5b59` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:41.326593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41907acf53304acd97a3ea430b8a5b59)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:41.328579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:41.461344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41907acf53304acd97a3ea430b8a5b59', 'request_id': 'req-ffdd8554-7797-4f0b-843f-0c461ed04725'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:41.550006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41907acf53304acd97a3ea430b8a5b59', 'request_id': 'req-ffdd8554-7797-4f0b-843f-0c461ed04725'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:41.560077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41907acf53304acd97a3ea430b8a5b59', 'request_id': 'req-ffdd8554-7797-4f0b-843f-0c461ed04725'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:41.584823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] The token cache is being invalidate because project 41907acf53304acd97a3ea430b8a5b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:41.585208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:41.586174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:41.588044 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1025/2091] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:41 2022] DELETE /identity/v3/projects/41907acf53304acd97a3ea430b8a5b59 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:41.592200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a07668-6362-415c-9399-2b195644ffe9 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:41.602356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a07668-6362-415c-9399-2b195644ffe9 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:41.621203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:41.663361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:41.663997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:41.664444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:41.664724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] PATH_INFO: `/v3/users/be1403f5de5f400f91cf0ccb441fa8ee` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:41.666065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1403f5de5f400f91cf0ccb441fa8ee)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:41.668046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:41.898891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1403f5de5f400f91cf0ccb441fa8ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:41.912022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1403f5de5f400f91cf0ccb441fa8ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:41.919915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1403f5de5f400f91cf0ccb441fa8ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:42.023624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1403f5de5f400f91cf0ccb441fa8ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:42.024271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0ead55df-629d-404d-856d-e91299abc2b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1403f5de5f400f91cf0ccb441fa8ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:42.031236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1067/2092] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:41 2022] DELETE /identity/v3/users/be1403f5de5f400f91cf0ccb441fa8ee => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:42.459642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:42.462353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:42.475658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd8554-7797-4f0b-843f-0c461ed04725 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:42.535530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:42.536155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:42.536433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:42.536732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] PATH_INFO: `/v3/projects/450e7628dea04032b32070b0a666033b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:42.537886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=450e7628dea04032b32070b0a666033b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:42.539805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:42.551919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:42.676780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '450e7628dea04032b32070b0a666033b', 'request_id': 'req-f759b9f4-28eb-460e-8e92-43cef8e9c130'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:42.789731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '450e7628dea04032b32070b0a666033b', 'request_id': 'req-f759b9f4-28eb-460e-8e92-43cef8e9c130'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:42.796970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '450e7628dea04032b32070b0a666033b', 'request_id': 'req-f759b9f4-28eb-460e-8e92-43cef8e9c130'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:42.808242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] The token cache is being invalidate because project 450e7628dea04032b32070b0a666033b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:42.808741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:42.810153 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f759b9f4-28eb-460e-8e92-43cef8e9c130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:42.817289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1026/2093] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:42 2022] DELETE /identity/v3/projects/450e7628dea04032b32070b0a666033b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:42.817373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230576-ab4a-43a8-b2c7-bf1536f9c8c0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:42.817373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230576-ab4a-43a8-b2c7-bf1536f9c8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:42.817373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230576-ab4a-43a8-b2c7-bf1536f9c8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:42.843465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e5230576-ab4a-43a8-b2c7-bf1536f9c8c0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:42.844718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e5230576-ab4a-43a8-b2c7-bf1536f9c8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:42.926138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1068/2094] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:42.928426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41a3f6-187e-4e33-b951-cb671e085f64 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:42.928685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41a3f6-187e-4e33-b951-cb671e085f64 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:42.928961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41a3f6-187e-4e33-b951-cb671e085f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:42.957776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ec41a3f6-187e-4e33-b951-cb671e085f64 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:42.959833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ec41a3f6-187e-4e33-b951-cb671e085f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:42.970901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1027/2095] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:42.975513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5230576-ab4a-43a8-b2c7-bf1536f9c8c0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:42.987591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5230576-ab4a-43a8-b2c7-bf1536f9c8c0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:43.000281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d56b3-3542-47e8-8d51-34035e3574ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:43.001034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d56b3-3542-47e8-8d51-34035e3574ab admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:43.001537 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d56b3-3542-47e8-8d51-34035e3574ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:43.001922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d56b3-3542-47e8-8d51-34035e3574ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:43.002649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d56b3-3542-47e8-8d51-34035e3574ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:43.003011 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d56b3-3542-47e8-8d51-34035e3574ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:43.004719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d56b3-3542-47e8-8d51-34035e3574ab admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:43.017350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1069/2096] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:32:43.021813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41a3f6-187e-4e33-b951-cb671e085f64 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:43.031486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41a3f6-187e-4e33-b951-cb671e085f64 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:43.036126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af011e8-1e31-46f1-aa6c-64bf69d72fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:43.041370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af011e8-1e31-46f1-aa6c-64bf69d72fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:43.041370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af011e8-1e31-46f1-aa6c-64bf69d72fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:43.041370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af011e8-1e31-46f1-aa6c-64bf69d72fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:43.041370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af011e8-1e31-46f1-aa6c-64bf69d72fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:43.041370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af011e8-1e31-46f1-aa6c-64bf69d72fdc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:43.156548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1028/2097] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:43 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:43.157817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d56b3-3542-47e8-8d51-34035e3574ab admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:43.169636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d56b3-3542-47e8-8d51-34035e3574ab admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:43.206853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:43.240716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04bb8f-e9ed-4d39-a698-06577009104e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:43.241373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04bb8f-e9ed-4d39-a698-06577009104e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:43.241680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04bb8f-e9ed-4d39-a698-06577009104e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:43.241960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04bb8f-e9ed-4d39-a698-06577009104e admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:43.244930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04bb8f-e9ed-4d39-a698-06577009104e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:43.244930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04bb8f-e9ed-4d39-a698-06577009104e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:43.336417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1070/2098] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:43 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:43.338160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af011e8-1e31-46f1-aa6c-64bf69d72fdc admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:43.349944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af011e8-1e31-46f1-aa6c-64bf69d72fdc admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:43.355471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e1c8c-c71d-4f2a-8f36-60389607d0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:43.356099 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e1c8c-c71d-4f2a-8f36-60389607d0fd admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:43.356376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e1c8c-c71d-4f2a-8f36-60389607d0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:43.357286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e1c8c-c71d-4f2a-8f36-60389607d0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:43.357286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e1c8c-c71d-4f2a-8f36-60389607d0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:43.358900 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e1c8c-c71d-4f2a-8f36-60389607d0fd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:43.369385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1029/2099] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:32:43.373427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04bb8f-e9ed-4d39-a698-06577009104e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:43.383306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04bb8f-e9ed-4d39-a698-06577009104e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:43.388974 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd96635-177c-4fbc-bc48-11888e397f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:43.389567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd96635-177c-4fbc-bc48-11888e397f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:43.390017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd96635-177c-4fbc-bc48-11888e397f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:43.393009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd96635-177c-4fbc-bc48-11888e397f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:43.393712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd96635-177c-4fbc-bc48-11888e397f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:43.395296 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd96635-177c-4fbc-bc48-11888e397f8b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:43.407200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1071/2100] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:43.410912 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e1c8c-c71d-4f2a-8f36-60389607d0fd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:43.421182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e1c8c-c71d-4f2a-8f36-60389607d0fd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:43.426849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93551b-d712-4d55-8af4-84bc49dfaa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:43.427495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93551b-d712-4d55-8af4-84bc49dfaa43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:43.427826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93551b-d712-4d55-8af4-84bc49dfaa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:43.428233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93551b-d712-4d55-8af4-84bc49dfaa43 admin admin] PATH_INFO: `/v3/projects/65b530c5dcd04eff86aa05a794b78da4/users/1b69c60664b74bf797d05ab18fad1998/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:43.446161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93551b-d712-4d55-8af4-84bc49dfaa43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b530c5dcd04eff86aa05a794b78da4, user_id=1b69c60664b74bf797d05ab18fad1998, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:43.568982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93551b-d712-4d55-8af4-84bc49dfaa43 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:43.599210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1030/2101] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:43 2022] PUT /identity/v3/projects/65b530c5dcd04eff86aa05a794b78da4/users/1b69c60664b74bf797d05ab18fad1998/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:43.610691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa10f94-f819-4129-888a-58fdcc4c04f8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:43.610979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa10f94-f819-4129-888a-58fdcc4c04f8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:43.611254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa10f94-f819-4129-888a-58fdcc4c04f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:43.640663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dfa10f94-f819-4129-888a-58fdcc4c04f8 None None] MFA Rules not processed for user `1b69c60664b74bf797d05ab18fad1998`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:43.641929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dfa10f94-f819-4129-888a-58fdcc4c04f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:43.668224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93551b-d712-4d55-8af4-84bc49dfaa43 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:43.670460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4d93551b-d712-4d55-8af4-84bc49dfaa43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:43.710735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1072/2102] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:43.724762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa10f94-f819-4129-888a-58fdcc4c04f8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:43.726625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dfa10f94-f819-4129-888a-58fdcc4c04f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:43.740661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93551b-d712-4d55-8af4-84bc49dfaa43 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:43.745584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:43.746239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:43.747420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:43.747699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] PATH_INFO: `/v3/users/1ba7119871bd4bbcb1ed4d9ec6a1db87` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:43.748853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ba7119871bd4bbcb1ed4d9ec6a1db87)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:43.750823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:43.786692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa10f94-f819-4129-888a-58fdcc4c04f8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:43.790577 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541b9ee-2de7-4d22-bb4f-4fe808f5b902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:43.791195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541b9ee-2de7-4d22-bb4f-4fe808f5b902 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:43.791475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541b9ee-2de7-4d22-bb4f-4fe808f5b902 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:43.791800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541b9ee-2de7-4d22-bb4f-4fe808f5b902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:43.800839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541b9ee-2de7-4d22-bb4f-4fe808f5b902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:43.802300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541b9ee-2de7-4d22-bb4f-4fe808f5b902 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:43.812434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1073/2103] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:32:43.960938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:43.967526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ba7119871bd4bbcb1ed4d9ec6a1db87'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:43.968367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ba7119871bd4bbcb1ed4d9ec6a1db87'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:44.064342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ba7119871bd4bbcb1ed4d9ec6a1db87'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:44.078349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ba7119871bd4bbcb1ed4d9ec6a1db87'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:44.085520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ba7119871bd4bbcb1ed4d9ec6a1db87'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:44.093110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1031/2104] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:43 2022] DELETE /identity/v3/users/1ba7119871bd4bbcb1ed4d9ec6a1db87 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:44.097947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a38de-9061-48fe-abc8-a6561a97d1cd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:44.098628 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a38de-9061-48fe-abc8-a6561a97d1cd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:44.098802 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a38de-9061-48fe-abc8-a6561a97d1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:44.119468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-031a38de-9061-48fe-abc8-a6561a97d1cd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:44.122069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-031a38de-9061-48fe-abc8-a6561a97d1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:44.166681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:44.185581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1074/2105] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:44.195516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:44.196949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:44.254971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bb67a-a720-4e98-868e-95b8db41d05c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:44.258655 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6df45-c48b-42c7-afda-1cc0d19c41d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:44.259247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6df45-c48b-42c7-afda-1cc0d19c41d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:44.259523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6df45-c48b-42c7-afda-1cc0d19c41d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:44.259796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6df45-c48b-42c7-afda-1cc0d19c41d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:44.269694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6df45-c48b-42c7-afda-1cc0d19c41d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:44.271123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6df45-c48b-42c7-afda-1cc0d19c41d9 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:44.281091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1032/2106] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:44.740886 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a38de-9061-48fe-abc8-a6561a97d1cd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:44.742336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-031a38de-9061-48fe-abc8-a6561a97d1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:44.760337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a38de-9061-48fe-abc8-a6561a97d1cd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:44.764096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:44.764737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:44.765084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:44.765432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] PATH_INFO: `/v3/projects/ff89d3eba9ae46cabc56a0a8d3d867a1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:44.767501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff89d3eba9ae46cabc56a0a8d3d867a1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:44.770584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:44.906868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff89d3eba9ae46cabc56a0a8d3d867a1', 'request_id': 'req-c03e7dc6-a2e8-42ef-b888-65d404692568'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:44.978261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff89d3eba9ae46cabc56a0a8d3d867a1', 'request_id': 'req-c03e7dc6-a2e8-42ef-b888-65d404692568'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:44.986268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff89d3eba9ae46cabc56a0a8d3d867a1', 'request_id': 'req-c03e7dc6-a2e8-42ef-b888-65d404692568'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:45.002199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] The token cache is being invalidate because project ff89d3eba9ae46cabc56a0a8d3d867a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:45.002550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:45.017702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c03e7dc6-a2e8-42ef-b888-65d404692568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:45.017702 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1075/2107] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:44 2022] DELETE /identity/v3/projects/ff89d3eba9ae46cabc56a0a8d3d867a1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:45.018140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0773-f3d8-41b3-9179-989618e64215 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.018140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0773-f3d8-41b3-9179-989618e64215 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.018140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0773-f3d8-41b3-9179-989618e64215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:45.032461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3a9e0773-f3d8-41b3-9179-989618e64215 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:45.033076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3a9e0773-f3d8-41b3-9179-989618e64215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:45.109986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1033/2108] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:45.119635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef0f61-c71e-4be3-a46a-a61bc9200a65 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.120070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef0f61-c71e-4be3-a46a-a61bc9200a65 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.120757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef0f61-c71e-4be3-a46a-a61bc9200a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:45.144080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-97ef0f61-c71e-4be3-a46a-a61bc9200a65 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:45.145339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-97ef0f61-c71e-4be3-a46a-a61bc9200a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:45.154643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1076/2109] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:45.159013 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e0773-f3d8-41b3-9179-989618e64215 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:45.169795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e0773-f3d8-41b3-9179-989618e64215 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:45.174825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d60c0-51f0-44e5-8006-e8a06c34f256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:45.175110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d60c0-51f0-44e5-8006-e8a06c34f256 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.175275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d60c0-51f0-44e5-8006-e8a06c34f256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.175545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d60c0-51f0-44e5-8006-e8a06c34f256 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:45.176224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d60c0-51f0-44e5-8006-e8a06c34f256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:45.176588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d60c0-51f0-44e5-8006-e8a06c34f256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:45.178177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d60c0-51f0-44e5-8006-e8a06c34f256 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:45.191491 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1034/2110] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:32:45.195948 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef0f61-c71e-4be3-a46a-a61bc9200a65 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:45.209835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef0f61-c71e-4be3-a46a-a61bc9200a65 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:45.212028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51de79-dbb2-435f-b527-90fc3edba1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:45.212635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51de79-dbb2-435f-b527-90fc3edba1e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.212907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51de79-dbb2-435f-b527-90fc3edba1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.213341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51de79-dbb2-435f-b527-90fc3edba1e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:45.214181 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51de79-dbb2-435f-b527-90fc3edba1e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:45.216713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51de79-dbb2-435f-b527-90fc3edba1e2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:45.294057 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1077/2111] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:45 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:45.298310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d60c0-51f0-44e5-8006-e8a06c34f256 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:45.310409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d60c0-51f0-44e5-8006-e8a06c34f256 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:45.357884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:45.370355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec202bd8-42b3-4c38-a61f-dc565d942555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:45.371014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec202bd8-42b3-4c38-a61f-dc565d942555 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.371911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec202bd8-42b3-4c38-a61f-dc565d942555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.372351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec202bd8-42b3-4c38-a61f-dc565d942555 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:45.373240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec202bd8-42b3-4c38-a61f-dc565d942555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:45.375323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec202bd8-42b3-4c38-a61f-dc565d942555 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:45.471419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1035/2112] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:45 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:45.479824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51de79-dbb2-435f-b527-90fc3edba1e2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:45.488025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51de79-dbb2-435f-b527-90fc3edba1e2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:45.492999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2984-4a4d-45e9-8af7-108bce5d8027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:45.493635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2984-4a4d-45e9-8af7-108bce5d8027 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.493914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2984-4a4d-45e9-8af7-108bce5d8027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.494189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa2984-4a4d-45e9-8af7-108bce5d8027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:45.494834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa2984-4a4d-45e9-8af7-108bce5d8027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:45.497182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa2984-4a4d-45e9-8af7-108bce5d8027 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:45.507545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1078/2113] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:45.511726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec202bd8-42b3-4c38-a61f-dc565d942555 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:45.522249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec202bd8-42b3-4c38-a61f-dc565d942555 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:45.527023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45a568-abf5-493f-8ac1-db38cc1883c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:45.527665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a568-abf5-493f-8ac1-db38cc1883c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.527938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a568-abf5-493f-8ac1-db38cc1883c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.528205 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a568-abf5-493f-8ac1-db38cc1883c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:45.528841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45a568-abf5-493f-8ac1-db38cc1883c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:45.530523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45a568-abf5-493f-8ac1-db38cc1883c0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:45.541095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1036/2114] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:32:45.546233 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2984-4a4d-45e9-8af7-108bce5d8027 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:45.556123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa2984-4a4d-45e9-8af7-108bce5d8027 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:45.561148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bdf87c-5e70-4d96-9ae9-1bb0923d54af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:45.561827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdf87c-5e70-4d96-9ae9-1bb0923d54af admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.562107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdf87c-5e70-4d96-9ae9-1bb0923d54af admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.562377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdf87c-5e70-4d96-9ae9-1bb0923d54af admin admin] PATH_INFO: `/v3/projects/84f869e4af1941feb4e8ef44fb286056/users/5ccf7923c1bc4c129ecf1b6fbe22212f/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:45.578791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bdf87c-5e70-4d96-9ae9-1bb0923d54af admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f869e4af1941feb4e8ef44fb286056, user_id=5ccf7923c1bc4c129ecf1b6fbe22212f, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:45.582667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bdf87c-5e70-4d96-9ae9-1bb0923d54af admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:45.677136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1079/2115] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:45 2022] PUT /identity/v3/projects/84f869e4af1941feb4e8ef44fb286056/users/5ccf7923c1bc4c129ecf1b6fbe22212f/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:45.695080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a6452-b31d-4310-9c43-0ee453b58956 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.695571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a6452-b31d-4310-9c43-0ee453b58956 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.695847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a6452-b31d-4310-9c43-0ee453b58956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:45.730128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-341a6452-b31d-4310-9c43-0ee453b58956 None None] MFA Rules not processed for user `5ccf7923c1bc4c129ecf1b6fbe22212f`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:45.732107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-341a6452-b31d-4310-9c43-0ee453b58956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:45.799922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1037/2116] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:32:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:45.812357 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bdf87c-5e70-4d96-9ae9-1bb0923d54af admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:45.814066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c4bdf87c-5e70-4d96-9ae9-1bb0923d54af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:45.908413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bdf87c-5e70-4d96-9ae9-1bb0923d54af admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:45.908413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07e85c-a542-4cc5-b03b-094765429fc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:45.908413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07e85c-a542-4cc5-b03b-094765429fc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.908413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07e85c-a542-4cc5-b03b-094765429fc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.908830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07e85c-a542-4cc5-b03b-094765429fc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:45.908830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07e85c-a542-4cc5-b03b-094765429fc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:45.908830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07e85c-a542-4cc5-b03b-094765429fc7 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:45.915388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1080/2117] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:55.910744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:55.972433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:07.689431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0204-b8ad-42b2-8940-1bb34fb232c4 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:07.690088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0204-b8ad-42b2-8940-1bb34fb232c4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:07.690088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0204-b8ad-42b2-8940-1bb34fb232c4 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:07.691603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1038/2118] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:33:07.702656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfba7b-ac5a-43c6-9f7b-1beb3f8bca11 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:07.703186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfba7b-ac5a-43c6-9f7b-1beb3f8bca11 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:07.703309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfba7b-ac5a-43c6-9f7b-1beb3f8bca11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:07.724673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4ecfba7b-ac5a-43c6-9f7b-1beb3f8bca11 None None] MFA Rules not processed for user `c11c7fb612ab40d88f526b8b38dbfb23`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:07.725956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4ecfba7b-ac5a-43c6-9f7b-1beb3f8bca11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:07.807404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1081/2119] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:33:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:07.819133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7102f5-3765-49da-a500-6ddc6299656d None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:07.819415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7102f5-3765-49da-a500-6ddc6299656d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:07.819684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7102f5-3765-49da-a500-6ddc6299656d None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:07.821076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1039/2120] 10.222.0.42 () {54 vars in 961 bytes} [Thu Sep 1 05:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:33:07.826222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e055e-73d3-4703-bf6f-6c055920bdca None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:07.826515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e055e-73d3-4703-bf6f-6c055920bdca None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:07.834361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e055e-73d3-4703-bf6f-6c055920bdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:07.846924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-575e055e-73d3-4703-bf6f-6c055920bdca None None] MFA Rules not processed for user `c11c7fb612ab40d88f526b8b38dbfb23`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:07.848136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-575e055e-73d3-4703-bf6f-6c055920bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:07.861432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1082/2121] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Sep 1 05:33:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:07.869271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7102f5-3765-49da-a500-6ddc6299656d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:07.872738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e055e-73d3-4703-bf6f-6c055920bdca None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:07.874982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-575e055e-73d3-4703-bf6f-6c055920bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:07.893125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7102f5-3765-49da-a500-6ddc6299656d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:07.900172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705882f-6195-4d6e-b3ff-1c598927821b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:07.901157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705882f-6195-4d6e-b3ff-1c598927821b service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:07.901587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705882f-6195-4d6e-b3ff-1c598927821b service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:07.902015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705882f-6195-4d6e-b3ff-1c598927821b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:07.913503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705882f-6195-4d6e-b3ff-1c598927821b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:07.915405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705882f-6195-4d6e-b3ff-1c598927821b service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:07.938175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1040/2122] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:07.967046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e055e-73d3-4703-bf6f-6c055920bdca None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:07.972754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:07.977901 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:07.978124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:07.978401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] PATH_INFO: `/v3/users/e529f3610d114981a5a3d37bdf832dab` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:07.979603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e529f3610d114981a5a3d37bdf832dab)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:07.981559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:08.156886 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e529f3610d114981a5a3d37bdf832dab'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:08.168807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e529f3610d114981a5a3d37bdf832dab'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:08.175698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e529f3610d114981a5a3d37bdf832dab'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:08.257007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e529f3610d114981a5a3d37bdf832dab'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:08.257668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-14e474aa-eab5-46b1-a94c-811ff8939bf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e529f3610d114981a5a3d37bdf832dab'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:08.264465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1083/2123] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:07 2022] DELETE /identity/v3/users/e529f3610d114981a5a3d37bdf832dab => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:08.508918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705882f-6195-4d6e-b3ff-1c598927821b service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:08.520716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705882f-6195-4d6e-b3ff-1c598927821b service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:08.587945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:08.589337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:08.589648 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:08.589910 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] PATH_INFO: `/v3/projects/9f237ba116b64751bee9817c19226b3f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:08.591067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f237ba116b64751bee9817c19226b3f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:08.593695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:08.693580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f237ba116b64751bee9817c19226b3f', 'request_id': 'req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:08.769285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f237ba116b64751bee9817c19226b3f', 'request_id': 'req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:08.776602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f237ba116b64751bee9817c19226b3f', 'request_id': 'req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:08.791884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] The token cache is being invalidate because project 9f237ba116b64751bee9817c19226b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:08.792240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:08.792968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6708660e-1ac7-45a0-b3cd-a264f3e1eaac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:08.798418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1041/2124] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:08 2022] DELETE /identity/v3/projects/9f237ba116b64751bee9817c19226b3f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:08.800235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb652b6-a214-44ad-86b3-c8e813342315 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:08.800703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb652b6-a214-44ad-86b3-c8e813342315 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:08.800818 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb652b6-a214-44ad-86b3-c8e813342315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:08.823509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0fb652b6-a214-44ad-86b3-c8e813342315 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:08.824738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0fb652b6-a214-44ad-86b3-c8e813342315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:08.829662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:08.896041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1084/2125] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:08.904037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20eeae-0141-4b74-a00f-310d50cc8cd5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:08.904353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20eeae-0141-4b74-a00f-310d50cc8cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:08.904675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20eeae-0141-4b74-a00f-310d50cc8cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:08.925715 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2e20eeae-0141-4b74-a00f-310d50cc8cd5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:08.927018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2e20eeae-0141-4b74-a00f-310d50cc8cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:08.939385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1042/2126] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:08.940509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb652b6-a214-44ad-86b3-c8e813342315 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:08.950229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb652b6-a214-44ad-86b3-c8e813342315 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:08.954843 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9961f-7495-40c8-bad6-ea92c51603fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:08.955441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9961f-7495-40c8-bad6-ea92c51603fa admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:08.955777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9961f-7495-40c8-bad6-ea92c51603fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:08.956059 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9961f-7495-40c8-bad6-ea92c51603fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:08.956699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9961f-7495-40c8-bad6-ea92c51603fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:33:08.957002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9961f-7495-40c8-bad6-ea92c51603fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:08.959605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9961f-7495-40c8-bad6-ea92c51603fa admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:08.971696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1085/2127] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:33:08.976003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20eeae-0141-4b74-a00f-310d50cc8cd5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:08.990167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20eeae-0141-4b74-a00f-310d50cc8cd5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:08.991263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fc713-6d80-4cdf-ae8d-234d7a43ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:08.991859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fc713-6d80-4cdf-ae8d-234d7a43ecf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:08.992134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fc713-6d80-4cdf-ae8d-234d7a43ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:08.992402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fc713-6d80-4cdf-ae8d-234d7a43ecf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:08.993033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fc713-6d80-4cdf-ae8d-234d7a43ecf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:08.994960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fc713-6d80-4cdf-ae8d-234d7a43ecf0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:09.087087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1043/2128] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:33:08 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:09.091277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9961f-7495-40c8-bad6-ea92c51603fa admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:09.101635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9961f-7495-40c8-bad6-ea92c51603fa admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:09.158682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc40623-9c7c-4255-8557-67959d1aa7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:09.159308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc40623-9c7c-4255-8557-67959d1aa7f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:09.159584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc40623-9c7c-4255-8557-67959d1aa7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:09.159851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc40623-9c7c-4255-8557-67959d1aa7f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:09.160464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc40623-9c7c-4255-8557-67959d1aa7f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:09.162431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc40623-9c7c-4255-8557-67959d1aa7f6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:09.256760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1086/2129] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:33:09 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:09.260802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fc713-6d80-4cdf-ae8d-234d7a43ecf0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:09.271922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fc713-6d80-4cdf-ae8d-234d7a43ecf0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:09.276634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a728a-d8d0-409d-a541-435c6f2f52c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:09.277274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a728a-d8d0-409d-a541-435c6f2f52c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:09.277591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a728a-d8d0-409d-a541-435c6f2f52c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:09.277968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a728a-d8d0-409d-a541-435c6f2f52c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:09.278610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a728a-d8d0-409d-a541-435c6f2f52c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:09.281256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a728a-d8d0-409d-a541-435c6f2f52c5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:09.291354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1044/2130] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:09.295402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc40623-9c7c-4255-8557-67959d1aa7f6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:09.306723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc40623-9c7c-4255-8557-67959d1aa7f6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:09.312006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e4a05-c1f5-45ee-b7fb-ee1d9d363f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:09.312728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e4a05-c1f5-45ee-b7fb-ee1d9d363f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:09.313005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e4a05-c1f5-45ee-b7fb-ee1d9d363f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:09.313278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e4a05-c1f5-45ee-b7fb-ee1d9d363f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:09.313960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e4a05-c1f5-45ee-b7fb-ee1d9d363f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:09.315606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e4a05-c1f5-45ee-b7fb-ee1d9d363f25 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:09.326412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1087/2131] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:09.331139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a728a-d8d0-409d-a541-435c6f2f52c5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:09.341162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a728a-d8d0-409d-a541-435c6f2f52c5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:09.345924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ed0b3-33e3-4616-a153-25ae34476db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:09.346568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ed0b3-33e3-4616-a153-25ae34476db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:09.346864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ed0b3-33e3-4616-a153-25ae34476db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:09.347119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ed0b3-33e3-4616-a153-25ae34476db8 admin admin] PATH_INFO: `/v3/projects/89a453fc88ee49548218764a8f837df9/users/d401159cac744c098c063fd3fe2f2fe2/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:09.362071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ed0b3-33e3-4616-a153-25ae34476db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a453fc88ee49548218764a8f837df9, user_id=d401159cac744c098c063fd3fe2f2fe2, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:09.364722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ed0b3-33e3-4616-a153-25ae34476db8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:09.432079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1045/2132] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:09 2022] PUT /identity/v3/projects/89a453fc88ee49548218764a8f837df9/users/d401159cac744c098c063fd3fe2f2fe2/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:09.444906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8da4-4ce9-4fde-ad1b-4692d51ff2c5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:09.445192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8da4-4ce9-4fde-ad1b-4692d51ff2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:09.445479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8da4-4ce9-4fde-ad1b-4692d51ff2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:09.474176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-96cb8da4-4ce9-4fde-ad1b-4692d51ff2c5 None None] MFA Rules not processed for user `d401159cac744c098c063fd3fe2f2fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:09.475394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-96cb8da4-4ce9-4fde-ad1b-4692d51ff2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:09.541949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1088/2133] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:09.546954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ed0b3-33e3-4616-a153-25ae34476db8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:09.560957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:09.610836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ed0b3-33e3-4616-a153-25ae34476db8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:09.615647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409156e8-9e5c-4e63-b235-b8b15edd6203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:09.616557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409156e8-9e5c-4e63-b235-b8b15edd6203 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:09.617021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409156e8-9e5c-4e63-b235-b8b15edd6203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:09.617366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409156e8-9e5c-4e63-b235-b8b15edd6203 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:09.618401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409156e8-9e5c-4e63-b235-b8b15edd6203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:09.620307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409156e8-9e5c-4e63-b235-b8b15edd6203 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:09.733751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1046/2134] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:33:09 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:09.738012 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb8da4-4ce9-4fde-ad1b-4692d51ff2c5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:09.745136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:09.750021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb8da4-4ce9-4fde-ad1b-4692d51ff2c5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:09.807267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a8bd9-92ae-4a60-9dc9-877b94fdfbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:09.807924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a8bd9-92ae-4a60-9dc9-877b94fdfbb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:09.808382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a8bd9-92ae-4a60-9dc9-877b94fdfbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:09.808655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a8bd9-92ae-4a60-9dc9-877b94fdfbb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:09.809368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a8bd9-92ae-4a60-9dc9-877b94fdfbb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:09.811968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a8bd9-92ae-4a60-9dc9-877b94fdfbb3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:09.941093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1089/2135] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:33:09 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:09.942321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409156e8-9e5c-4e63-b235-b8b15edd6203 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:09.953816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409156e8-9e5c-4e63-b235-b8b15edd6203 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:09.958548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3628c8a-c46c-4f53-9cb0-ba07e3c8389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:09.959127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3628c8a-c46c-4f53-9cb0-ba07e3c8389d admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:09.959401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3628c8a-c46c-4f53-9cb0-ba07e3c8389d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:09.959668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3628c8a-c46c-4f53-9cb0-ba07e3c8389d admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:09.960302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3628c8a-c46c-4f53-9cb0-ba07e3c8389d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:09.961993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3628c8a-c46c-4f53-9cb0-ba07e3c8389d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:09.973757 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1047/2136] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:09.977851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a8bd9-92ae-4a60-9dc9-877b94fdfbb3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:09.988244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a8bd9-92ae-4a60-9dc9-877b94fdfbb3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:09.992792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fd38e-0504-445c-9a1b-ce555bad7681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:09.993437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fd38e-0504-445c-9a1b-ce555bad7681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:09.993756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fd38e-0504-445c-9a1b-ce555bad7681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:09.994027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fd38e-0504-445c-9a1b-ce555bad7681 admin admin] PATH_INFO: `/v3/projects/2c7d976754f64c95806a9ec17ba0ec8f/users/088958368fea4a3bb0ec87b9e95e0d5c/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:10.009925 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fd38e-0504-445c-9a1b-ce555bad7681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7d976754f64c95806a9ec17ba0ec8f, user_id=088958368fea4a3bb0ec87b9e95e0d5c, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:10.012583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fd38e-0504-445c-9a1b-ce555bad7681 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:10.103596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1090/2137] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:09 2022] PUT /identity/v3/projects/2c7d976754f64c95806a9ec17ba0ec8f/users/088958368fea4a3bb0ec87b9e95e0d5c/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:10.105740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3628c8a-c46c-4f53-9cb0-ba07e3c8389d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:10.123417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3628c8a-c46c-4f53-9cb0-ba07e3c8389d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:10.178244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aaf44-daa2-46e5-a72f-af140cb0992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:10.178853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aaf44-daa2-46e5-a72f-af140cb0992c admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:10.179159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aaf44-daa2-46e5-a72f-af140cb0992c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:10.179431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aaf44-daa2-46e5-a72f-af140cb0992c admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:10.180070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541aaf44-daa2-46e5-a72f-af140cb0992c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:10.181955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541aaf44-daa2-46e5-a72f-af140cb0992c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:10.192439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1048/2138] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:10.196465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fd38e-0504-445c-9a1b-ce555bad7681 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:10.208329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fd38e-0504-445c-9a1b-ce555bad7681 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:10.213360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62a4b-c721-45b0-89e0-69c4a8c08c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:10.214083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62a4b-c721-45b0-89e0-69c4a8c08c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:10.214359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62a4b-c721-45b0-89e0-69c4a8c08c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:10.214625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62a4b-c721-45b0-89e0-69c4a8c08c5b admin admin] PATH_INFO: `/v3/projects/2c7d976754f64c95806a9ec17ba0ec8f/users/088958368fea4a3bb0ec87b9e95e0d5c/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:10.216513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62a4b-c721-45b0-89e0-69c4a8c08c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7d976754f64c95806a9ec17ba0ec8f, user_id=088958368fea4a3bb0ec87b9e95e0d5c, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:10.219430 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62a4b-c721-45b0-89e0-69c4a8c08c5b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:10.229813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1091/2139] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:10 2022] PUT /identity/v3/projects/2c7d976754f64c95806a9ec17ba0ec8f/users/088958368fea4a3bb0ec87b9e95e0d5c/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:10.240993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99bc42-2912-40e7-b45c-e23f07465aff None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:10.241280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99bc42-2912-40e7-b45c-e23f07465aff None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:10.241593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99bc42-2912-40e7-b45c-e23f07465aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:10.272289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4d99bc42-2912-40e7-b45c-e23f07465aff None None] MFA Rules not processed for user `088958368fea4a3bb0ec87b9e95e0d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:10.273515 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4d99bc42-2912-40e7-b45c-e23f07465aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:10.347148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1049/2140] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:10.362197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62a4b-c721-45b0-89e0-69c4a8c08c5b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:10.363982 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2fc62a4b-c721-45b0-89e0-69c4a8c08c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:10.425902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62a4b-c721-45b0-89e0-69c4a8c08c5b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:10.430000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbb20f-feae-465f-8991-12ced9248492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:10.430727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbb20f-feae-465f-8991-12ced9248492 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:10.431381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbb20f-feae-465f-8991-12ced9248492 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:10.431751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbb20f-feae-465f-8991-12ced9248492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:10.442190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbb20f-feae-465f-8991-12ced9248492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:10.443887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbb20f-feae-465f-8991-12ced9248492 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:10.453829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1092/2141] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:10.710675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99bc42-2912-40e7-b45c-e23f07465aff None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:10.721876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99bc42-2912-40e7-b45c-e23f07465aff None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:10.725396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb778cfa-69c3-4cdc-97fe-9ad9fce2503a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:10.726028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb778cfa-69c3-4cdc-97fe-9ad9fce2503a service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:10.726303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb778cfa-69c3-4cdc-97fe-9ad9fce2503a service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:10.726570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb778cfa-69c3-4cdc-97fe-9ad9fce2503a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:10.732076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:10.784856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb778cfa-69c3-4cdc-97fe-9ad9fce2503a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:10.786319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb778cfa-69c3-4cdc-97fe-9ad9fce2503a service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:10.798774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1050/2142] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:20.449268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:20.768522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992ea83-908f-4e3c-8b50-a02741c1acbd None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:20.768890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992ea83-908f-4e3c-8b50-a02741c1acbd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:20.769162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992ea83-908f-4e3c-8b50-a02741c1acbd None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:20.770610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1093/2143] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:33:20.803686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d20868-f1b5-450c-93d3-60c611ce7f86 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:20.804234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d20868-f1b5-450c-93d3-60c611ce7f86 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:20.804234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d20868-f1b5-450c-93d3-60c611ce7f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:20.824733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-40d20868-f1b5-450c-93d3-60c611ce7f86 None None] MFA Rules not processed for user `c11c7fb612ab40d88f526b8b38dbfb23`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:20.825979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-40d20868-f1b5-450c-93d3-60c611ce7f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:20.892295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1051/2144] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:33:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:20.907966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992ea83-908f-4e3c-8b50-a02741c1acbd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:20.909976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f992ea83-908f-4e3c-8b50-a02741c1acbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:20.928845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992ea83-908f-4e3c-8b50-a02741c1acbd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:20.934713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e0117-277b-41f6-be40-c17895d8ee1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:20.936528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e0117-277b-41f6-be40-c17895d8ee1b service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:20.936528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e0117-277b-41f6-be40-c17895d8ee1b service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:20.936528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e0117-277b-41f6-be40-c17895d8ee1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:20.947080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e0117-277b-41f6-be40-c17895d8ee1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:20.949003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e0117-277b-41f6-be40-c17895d8ee1b service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:20.960599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1094/2145] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:21.530129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d20868-f1b5-450c-93d3-60c611ce7f86 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:21.531721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-40d20868-f1b5-450c-93d3-60c611ce7f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:21.600261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d20868-f1b5-450c-93d3-60c611ce7f86 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:21.604873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:21.605520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:21.605795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:21.606063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] PATH_INFO: `/v3/users/19fd2c60d6c24b18b3b93b4b67729589` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:21.607176 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19fd2c60d6c24b18b3b93b4b67729589)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:21.610064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:21.787419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:21.788966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19fd2c60d6c24b18b3b93b4b67729589'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:21.789649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19fd2c60d6c24b18b3b93b4b67729589'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:21.890882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19fd2c60d6c24b18b3b93b4b67729589'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:21.903914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19fd2c60d6c24b18b3b93b4b67729589'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:21.911848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19fd2c60d6c24b18b3b93b4b67729589'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:21.918410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1052/2146] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:21 2022] DELETE /identity/v3/users/19fd2c60d6c24b18b3b93b4b67729589 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:22.190332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e0117-277b-41f6-be40-c17895d8ee1b service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:22.202140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e0117-277b-41f6-be40-c17895d8ee1b service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:22.264431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:22.265154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:22.265564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:22.265742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] PATH_INFO: `/v3/projects/6f901db083e7435ba2f1c2b03febeb1f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:22.266942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f901db083e7435ba2f1c2b03febeb1f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:22.268895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:22.389480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f901db083e7435ba2f1c2b03febeb1f', 'request_id': 'req-0667d91a-6ebb-45b0-a596-899d66e425c3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:22.465234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f901db083e7435ba2f1c2b03febeb1f', 'request_id': 'req-0667d91a-6ebb-45b0-a596-899d66e425c3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:22.474767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f901db083e7435ba2f1c2b03febeb1f', 'request_id': 'req-0667d91a-6ebb-45b0-a596-899d66e425c3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:22.487635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] The token cache is being invalidate because project 6f901db083e7435ba2f1c2b03febeb1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:22.488283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:22.488801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:22.493290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1095/2147] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:22 2022] DELETE /identity/v3/projects/6f901db083e7435ba2f1c2b03febeb1f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:22.494584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:22.496208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:22.509469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c1112-c766-46aa-a9d0-180331ef04a2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:22.567809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:22.568429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:22.568707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:22.568978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] PATH_INFO: `/v3/users/9d4dbadb297440689aca26af6aab2344` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:22.570114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d4dbadb297440689aca26af6aab2344)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:22.572045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:22.770465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d4dbadb297440689aca26af6aab2344'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:22.775223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d4dbadb297440689aca26af6aab2344'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:22.871055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d4dbadb297440689aca26af6aab2344'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:22.884656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d4dbadb297440689aca26af6aab2344'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:22.891817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d4dbadb297440689aca26af6aab2344'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:22.897967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1053/2148] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:22 2022] DELETE /identity/v3/users/9d4dbadb297440689aca26af6aab2344 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:23.184028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:23.186230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:23.204814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667d91a-6ebb-45b0-a596-899d66e425c3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:23.265242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:23.266202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:23.266480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:23.266833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] PATH_INFO: `/v3/projects/5aee6445ce014335a4073ddb51dd1382` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:23.268408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aee6445ce014335a4073ddb51dd1382)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:23.270387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:23.294640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:23.436073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aee6445ce014335a4073ddb51dd1382', 'request_id': 'req-0e9e27c9-633c-4631-964a-7f2ac6f65b62'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.525991 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:23.527578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:23.533990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aee6445ce014335a4073ddb51dd1382', 'request_id': 'req-0e9e27c9-633c-4631-964a-7f2ac6f65b62'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.541629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aee6445ce014335a4073ddb51dd1382', 'request_id': 'req-0e9e27c9-633c-4631-964a-7f2ac6f65b62'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.548468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2debcff-8967-4f2c-a6a3-11c0393d0aef admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:23.555034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] The token cache is being invalidate because project 5aee6445ce014335a4073ddb51dd1382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:23.556037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.557122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e9e27c9-633c-4631-964a-7f2ac6f65b62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.559057 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1096/2149] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:23 2022] DELETE /identity/v3/projects/5aee6445ce014335a4073ddb51dd1382 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:23.565369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbdc07-aff9-4105-9f84-4ae42f7dc184 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:23.565881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbdc07-aff9-4105-9f84-4ae42f7dc184 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:23.566397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbdc07-aff9-4105-9f84-4ae42f7dc184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:23.588260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ccdbdc07-aff9-4105-9f84-4ae42f7dc184 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:23.589477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ccdbdc07-aff9-4105-9f84-4ae42f7dc184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:23.607252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:23.625527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:23.626161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:23.626434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:23.626701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] PATH_INFO: `/v3/users/5ccf7923c1bc4c129ecf1b6fbe22212f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:23.627805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ccf7923c1bc4c129ecf1b6fbe22212f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:23.629762 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:23.652579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1097/2150] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:23.660931 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c96d37-30a3-4b3d-bd0e-2f761d606c21 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:23.661211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c96d37-30a3-4b3d-bd0e-2f761d606c21 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:23.661510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c96d37-30a3-4b3d-bd0e-2f761d606c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:23.681359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-08c96d37-30a3-4b3d-bd0e-2f761d606c21 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:23.682601 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-08c96d37-30a3-4b3d-bd0e-2f761d606c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:23.691071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1098/2151] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:23.691867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c96d37-30a3-4b3d-bd0e-2f761d606c21 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:23.693055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-08c96d37-30a3-4b3d-bd0e-2f761d606c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:23.705799 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c96d37-30a3-4b3d-bd0e-2f761d606c21 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:23.710411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:23.710990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:23.711268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:23.711541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] PATH_INFO: `/v3/users/1b69c60664b74bf797d05ab18fad1998` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:23.712768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b69c60664b74bf797d05ab18fad1998)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:23.714707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:23.871511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ccf7923c1bc4c129ecf1b6fbe22212f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.872468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ccf7923c1bc4c129ecf1b6fbe22212f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.957130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b69c60664b74bf797d05ab18fad1998'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.959796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ccf7923c1bc4c129ecf1b6fbe22212f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.972626 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b69c60664b74bf797d05ab18fad1998'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.974909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ccf7923c1bc4c129ecf1b6fbe22212f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.981829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b69c60664b74bf797d05ab18fad1998'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.985026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ccf7923c1bc4c129ecf1b6fbe22212f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:23.991893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1054/2152] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:23 2022] DELETE /identity/v3/users/5ccf7923c1bc4c129ecf1b6fbe22212f => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:23.995551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:23.996923 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:24.061161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df2ff-285e-45b1-b226-ad25ab5dac61 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:24.065651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351dbbb-642a-4654-8baa-6ca2d23f4f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:24.066258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351dbbb-642a-4654-8baa-6ca2d23f4f64 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.066535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351dbbb-642a-4654-8baa-6ca2d23f4f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.066809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351dbbb-642a-4654-8baa-6ca2d23f4f64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.067452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351dbbb-642a-4654-8baa-6ca2d23f4f64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:33:24.067754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351dbbb-642a-4654-8baa-6ca2d23f4f64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:24.068160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b69c60664b74bf797d05ab18fad1998'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:24.069045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91bb7031-1b12-4409-81d8-55136cff8875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b69c60664b74bf797d05ab18fad1998'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:24.069359 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351dbbb-642a-4654-8baa-6ca2d23f4f64 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:24.076148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1099/2153] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:23 2022] DELETE /identity/v3/users/1b69c60664b74bf797d05ab18fad1998 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:24.077648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b52c-fe28-41d2-9bbf-124e5a508ed1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.077769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b52c-fe28-41d2-9bbf-124e5a508ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.078043 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b52c-fe28-41d2-9bbf-124e5a508ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.087483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1055/2154] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:33:24.087483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434df64-d146-447a-b3cd-a65fc12e7dd8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.087483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434df64-d146-447a-b3cd-a65fc12e7dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.087483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434df64-d146-447a-b3cd-a65fc12e7dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.102849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8334b52c-fe28-41d2-9bbf-124e5a508ed1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:24.104062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8334b52c-fe28-41d2-9bbf-124e5a508ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:24.109450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0434df64-d146-447a-b3cd-a65fc12e7dd8 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:24.110669 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0434df64-d146-447a-b3cd-a65fc12e7dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:24.115033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:24.159600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:24.166261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1100/2155] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:24.167076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334b52c-fe28-41d2-9bbf-124e5a508ed1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:24.173589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1056/2156] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:24.176470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434df64-d146-447a-b3cd-a65fc12e7dd8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:24.176597 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8334b52c-fe28-41d2-9bbf-124e5a508ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:24.177845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0434df64-d146-447a-b3cd-a65fc12e7dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:24.194163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334b52c-fe28-41d2-9bbf-124e5a508ed1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:24.199417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7a60c-27d3-4def-b0cf-4fe80f54346e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:24.200028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7a60c-27d3-4def-b0cf-4fe80f54346e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.200311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7a60c-27d3-4def-b0cf-4fe80f54346e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.200587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7a60c-27d3-4def-b0cf-4fe80f54346e admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.201443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7a60c-27d3-4def-b0cf-4fe80f54346e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:24.204201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7a60c-27d3-4def-b0cf-4fe80f54346e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:24.243250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434df64-d146-447a-b3cd-a65fc12e7dd8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:24.246927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7e4e3-05d5-4c00-9e26-7f7264deb09e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:24.247527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e4e3-05d5-4c00-9e26-7f7264deb09e service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.248340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e4e3-05d5-4c00-9e26-7f7264deb09e service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.248803 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e4e3-05d5-4c00-9e26-7f7264deb09e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.257776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7e4e3-05d5-4c00-9e26-7f7264deb09e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:24.259199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7e4e3-05d5-4c00-9e26-7f7264deb09e service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:24.269320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1057/2157] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:24.270144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7e4e3-05d5-4c00-9e26-7f7264deb09e service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:24.277874 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7e4e3-05d5-4c00-9e26-7f7264deb09e service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:24.281909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4ef4b-a9af-4967-aa91-16eba55bdfb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:24.282473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4ef4b-a9af-4967-aa91-16eba55bdfb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.282747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4ef4b-a9af-4967-aa91-16eba55bdfb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.283011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4ef4b-a9af-4967-aa91-16eba55bdfb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.292021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4ef4b-a9af-4967-aa91-16eba55bdfb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:24.293498 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4ef4b-a9af-4967-aa91-16eba55bdfb7 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:24.303708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1058/2158] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:24.314114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1101/2159] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:33:24 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:24.318173 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4ef4b-a9af-4967-aa91-16eba55bdfb7 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:24.328992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4ef4b-a9af-4967-aa91-16eba55bdfb7 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:24.391842 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460733db-36bb-4e7c-be27-1b56b134f85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:24.392456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460733db-36bb-4e7c-be27-1b56b134f85b admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.392729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460733db-36bb-4e7c-be27-1b56b134f85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.392995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460733db-36bb-4e7c-be27-1b56b134f85b admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.393775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460733db-36bb-4e7c-be27-1b56b134f85b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:24.396179 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460733db-36bb-4e7c-be27-1b56b134f85b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:24.496026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1059/2160] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:33:24 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:33:24.500400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7a60c-27d3-4def-b0cf-4fe80f54346e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:24.511508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7a60c-27d3-4def-b0cf-4fe80f54346e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:24.520761 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd095da-b380-4670-be46-9e66221f11dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:24.524467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd095da-b380-4670-be46-9e66221f11dd admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.524467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd095da-b380-4670-be46-9e66221f11dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.524467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd095da-b380-4670-be46-9e66221f11dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.524467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd095da-b380-4670-be46-9e66221f11dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:24.524979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd095da-b380-4670-be46-9e66221f11dd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:24.537513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1102/2161] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:24.540445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460733db-36bb-4e7c-be27-1b56b134f85b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:24.551711 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460733db-36bb-4e7c-be27-1b56b134f85b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:24.556964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54557bab-9567-4ee4-8b18-8ce6eebb9931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:24.557805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54557bab-9567-4ee4-8b18-8ce6eebb9931 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.558138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54557bab-9567-4ee4-8b18-8ce6eebb9931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.558413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54557bab-9567-4ee4-8b18-8ce6eebb9931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.559096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54557bab-9567-4ee4-8b18-8ce6eebb9931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:24.560798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54557bab-9567-4ee4-8b18-8ce6eebb9931 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:24.574294 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1060/2162] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:24.575480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd095da-b380-4670-be46-9e66221f11dd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:24.586792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd095da-b380-4670-be46-9e66221f11dd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:24.593732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54eebb-028d-4758-ab3d-8f300764e79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:24.593732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54eebb-028d-4758-ab3d-8f300764e79b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.593732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54eebb-028d-4758-ab3d-8f300764e79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.594056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54eebb-028d-4758-ab3d-8f300764e79b admin admin] PATH_INFO: `/v3/projects/ff7b630ffa4f429498fe145ce35ca4d3/users/a52c2a9fc56144d9bfaf4c18566b8096/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.657760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54eebb-028d-4758-ab3d-8f300764e79b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7b630ffa4f429498fe145ce35ca4d3, user_id=a52c2a9fc56144d9bfaf4c18566b8096, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:24.660735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54eebb-028d-4758-ab3d-8f300764e79b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:24.736369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1103/2163] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:24 2022] PUT /identity/v3/projects/ff7b630ffa4f429498fe145ce35ca4d3/users/a52c2a9fc56144d9bfaf4c18566b8096/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:24.746555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1d759-b76d-4184-a808-bd6b6bf364d7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.746842 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1d759-b76d-4184-a808-bd6b6bf364d7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.747118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1d759-b76d-4184-a808-bd6b6bf364d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.777273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-21c1d759-b76d-4184-a808-bd6b6bf364d7 None None] MFA Rules not processed for user `a52c2a9fc56144d9bfaf4c18566b8096`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:24.778568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-21c1d759-b76d-4184-a808-bd6b6bf364d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:24.841886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:24.852306 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1061/2164] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:24.856753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54eebb-028d-4758-ab3d-8f300764e79b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:24.909004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1d759-b76d-4184-a808-bd6b6bf364d7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:24.910898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-21c1d759-b76d-4184-a808-bd6b6bf364d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:24.925785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54eebb-028d-4758-ab3d-8f300764e79b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:24.927712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1d759-b76d-4184-a808-bd6b6bf364d7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:24.934666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea583a-1262-41b5-877b-e2468bca3052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:24.934666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea583a-1262-41b5-877b-e2468bca3052 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.934860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:24.934968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea583a-1262-41b5-877b-e2468bca3052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.934968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea583a-1262-41b5-877b-e2468bca3052 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.935696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:24.935696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:24.936320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] PATH_INFO: `/v3/projects/84f869e4af1941feb4e8ef44fb286056` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:24.937581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f869e4af1941feb4e8ef44fb286056)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:24.942182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:24.942417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea583a-1262-41b5-877b-e2468bca3052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:24.944169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea583a-1262-41b5-877b-e2468bca3052 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:25.046466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1104/2165] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:33:24 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:25.047679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea583a-1262-41b5-877b-e2468bca3052 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:25.051054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2eea583a-1262-41b5-877b-e2468bca3052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:25.057780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f869e4af1941feb4e8ef44fb286056', 'request_id': 'req-79f7082e-f6b3-49bc-9d30-ec500d2c5146'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:25.097402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:25.116330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea583a-1262-41b5-877b-e2468bca3052 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:25.121124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:25.121790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:25.122073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:25.122426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] PATH_INFO: `/v3/projects/65b530c5dcd04eff86aa05a794b78da4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:25.123867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b530c5dcd04eff86aa05a794b78da4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:25.125829 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:25.147718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f869e4af1941feb4e8ef44fb286056', 'request_id': 'req-79f7082e-f6b3-49bc-9d30-ec500d2c5146'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:25.155968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f869e4af1941feb4e8ef44fb286056', 'request_id': 'req-79f7082e-f6b3-49bc-9d30-ec500d2c5146'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:25.166625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] The token cache is being invalidate because project 84f869e4af1941feb4e8ef44fb286056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:25.166912 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:25.167582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:25.169569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1062/2166] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:24 2022] DELETE /identity/v3/projects/84f869e4af1941feb4e8ef44fb286056 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:25.170321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:25.171583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:25.218466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b530c5dcd04eff86aa05a794b78da4', 'request_id': 'req-b6815d08-611e-440d-adb3-dceb2aad7325'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:25.244611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7082e-f6b3-49bc-9d30-ec500d2c5146 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:25.291230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b530c5dcd04eff86aa05a794b78da4', 'request_id': 'req-b6815d08-611e-440d-adb3-dceb2aad7325'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:25.300339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b530c5dcd04eff86aa05a794b78da4', 'request_id': 'req-b6815d08-611e-440d-adb3-dceb2aad7325'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:25.312039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b331ae-713f-48bc-bfde-eeaf0d4dbaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:25.312666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b331ae-713f-48bc-bfde-eeaf0d4dbaa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:25.312948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b331ae-713f-48bc-bfde-eeaf0d4dbaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:25.313154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] The token cache is being invalidate because project 65b530c5dcd04eff86aa05a794b78da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:25.313277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b331ae-713f-48bc-bfde-eeaf0d4dbaa7 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:25.313475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:25.313907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b331ae-713f-48bc-bfde-eeaf0d4dbaa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:25.314144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:25.316247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1105/2167] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:25 2022] DELETE /identity/v3/projects/65b530c5dcd04eff86aa05a794b78da4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:25.316374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b331ae-713f-48bc-bfde-eeaf0d4dbaa7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:25.434943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1063/2168] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:33:25 2022] POST /identity/v3/users => generated 298 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:25.440413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:25.441863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:25.455824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6815d08-611e-440d-adb3-dceb2aad7325 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:25.462142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42599ced-80e0-43c7-bc47-509484091d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:25.462777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42599ced-80e0-43c7-bc47-509484091d3b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:25.463054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42599ced-80e0-43c7-bc47-509484091d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:25.463325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42599ced-80e0-43c7-bc47-509484091d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:25.463984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42599ced-80e0-43c7-bc47-509484091d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:25.465649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42599ced-80e0-43c7-bc47-509484091d3b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:25.479570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1106/2169] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:25.486010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b331ae-713f-48bc-bfde-eeaf0d4dbaa7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:25.497290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b331ae-713f-48bc-bfde-eeaf0d4dbaa7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:25.507261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd2061-1bd1-4f05-bd3e-0acb01bd0918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:25.508023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd2061-1bd1-4f05-bd3e-0acb01bd0918 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:25.508365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd2061-1bd1-4f05-bd3e-0acb01bd0918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:25.508640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd2061-1bd1-4f05-bd3e-0acb01bd0918 admin admin] PATH_INFO: `/v3/projects/397b6b3a855447ce8acbe5ac42ec4ddd/users/2660f32b04194a14a82ef2571e73d9cd/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:25.521807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:25.525850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd2061-1bd1-4f05-bd3e-0acb01bd0918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=397b6b3a855447ce8acbe5ac42ec4ddd, user_id=2660f32b04194a14a82ef2571e73d9cd, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:25.528493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd2061-1bd1-4f05-bd3e-0acb01bd0918 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:25.593721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1064/2170] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:25 2022] PUT /identity/v3/projects/397b6b3a855447ce8acbe5ac42ec4ddd/users/2660f32b04194a14a82ef2571e73d9cd/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:25.597629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42599ced-80e0-43c7-bc47-509484091d3b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:25.608333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42599ced-80e0-43c7-bc47-509484091d3b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:25.666150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2a679-32a3-4f44-a77e-73191f823806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:25.666760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2a679-32a3-4f44-a77e-73191f823806 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:25.667042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2a679-32a3-4f44-a77e-73191f823806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:25.667315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2a679-32a3-4f44-a77e-73191f823806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:25.668026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2a679-32a3-4f44-a77e-73191f823806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:25.670292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2a679-32a3-4f44-a77e-73191f823806 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:25.680410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1107/2171] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:25.684808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd2061-1bd1-4f05-bd3e-0acb01bd0918 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:25.695524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd2061-1bd1-4f05-bd3e-0acb01bd0918 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:25.700015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36cdea-6a8d-4906-b269-28d8bce43708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:25.700857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36cdea-6a8d-4906-b269-28d8bce43708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:25.701033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36cdea-6a8d-4906-b269-28d8bce43708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:25.701536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36cdea-6a8d-4906-b269-28d8bce43708 admin admin] PATH_INFO: `/v3/projects/397b6b3a855447ce8acbe5ac42ec4ddd/users/2660f32b04194a14a82ef2571e73d9cd/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:25.703446 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36cdea-6a8d-4906-b269-28d8bce43708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=397b6b3a855447ce8acbe5ac42ec4ddd, user_id=2660f32b04194a14a82ef2571e73d9cd, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:25.706215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36cdea-6a8d-4906-b269-28d8bce43708 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:25.725030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1065/2172] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:25 2022] PUT /identity/v3/projects/397b6b3a855447ce8acbe5ac42ec4ddd/users/2660f32b04194a14a82ef2571e73d9cd/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:25.736901 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e505b-9a74-4c34-af2a-3e385622fbcf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:25.737182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e505b-9a74-4c34-af2a-3e385622fbcf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:25.737474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e505b-9a74-4c34-af2a-3e385622fbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:25.766672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ee6e505b-9a74-4c34-af2a-3e385622fbcf None None] MFA Rules not processed for user `2660f32b04194a14a82ef2571e73d9cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:25.767881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ee6e505b-9a74-4c34-af2a-3e385622fbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:25.815193 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:25.839965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1108/2173] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:25.851619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36cdea-6a8d-4906-b269-28d8bce43708 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:25.853793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5a36cdea-6a8d-4906-b269-28d8bce43708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:25.916540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36cdea-6a8d-4906-b269-28d8bce43708 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:25.920262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c86242-ec15-4a80-aa6d-87475880590f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:25.920857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c86242-ec15-4a80-aa6d-87475880590f service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:25.921138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c86242-ec15-4a80-aa6d-87475880590f service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:25.921410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c86242-ec15-4a80-aa6d-87475880590f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:25.923164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b0c86242-ec15-4a80-aa6d-87475880590f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:25.981247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c86242-ec15-4a80-aa6d-87475880590f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:25.982944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c86242-ec15-4a80-aa6d-87475880590f service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:25.991826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1066/2174] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:35.910687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:35.993373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=57,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:37.784932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e505b-9a74-4c34-af2a-3e385622fbcf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:37.786389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ee6e505b-9a74-4c34-af2a-3e385622fbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:37.852418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e505b-9a74-4c34-af2a-3e385622fbcf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:37.857155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:37.857833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:37.858141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:37.858414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] PATH_INFO: `/v3/users/d401159cac744c098c063fd3fe2f2fe2` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:37.860074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d401159cac744c098c063fd3fe2f2fe2)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:37.862230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:38.031159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd401159cac744c098c063fd3fe2f2fe2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:38.047261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd401159cac744c098c063fd3fe2f2fe2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:38.054955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd401159cac744c098c063fd3fe2f2fe2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:38.121379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd401159cac744c098c063fd3fe2f2fe2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:38.122038 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd401159cac744c098c063fd3fe2f2fe2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:38.128794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1109/2175] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:37 2022] DELETE /identity/v3/users/d401159cac744c098c063fd3fe2f2fe2 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:38.133333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0aab1-e45a-4831-9806-bd84655b4e37 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:38.133882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0aab1-e45a-4831-9806-bd84655b4e37 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:38.134008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0aab1-e45a-4831-9806-bd84655b4e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:38.153598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9ed0aab1-e45a-4831-9806-bd84655b4e37 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:38.154892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9ed0aab1-e45a-4831-9806-bd84655b4e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:38.219558 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1067/2176] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:38.229710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:38.231448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:38.290162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ae12e-d54e-455c-83b2-3720a7246c03 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:38.293865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ddc24-3fbd-448d-a1d5-45ba12dabd04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:38.294467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ddc24-3fbd-448d-a1d5-45ba12dabd04 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:38.294748 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ddc24-3fbd-448d-a1d5-45ba12dabd04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:38.295022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ddc24-3fbd-448d-a1d5-45ba12dabd04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:38.303926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ddc24-3fbd-448d-a1d5-45ba12dabd04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:38.305636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ddc24-3fbd-448d-a1d5-45ba12dabd04 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:38.315493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1110/2177] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:33:38.931202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0aab1-e45a-4831-9806-bd84655b4e37 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:38.932848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9ed0aab1-e45a-4831-9806-bd84655b4e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:38.945663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0aab1-e45a-4831-9806-bd84655b4e37 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:38.958140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:38.958836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:38.959055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:38.959329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] PATH_INFO: `/v3/projects/89a453fc88ee49548218764a8f837df9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:38.960520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89a453fc88ee49548218764a8f837df9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:38.962803 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:39.071393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a453fc88ee49548218764a8f837df9', 'request_id': 'req-910e2ad6-542b-4952-ba1b-b21ce38dc003'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:39.135786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a453fc88ee49548218764a8f837df9', 'request_id': 'req-910e2ad6-542b-4952-ba1b-b21ce38dc003'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:39.144382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a453fc88ee49548218764a8f837df9', 'request_id': 'req-910e2ad6-542b-4952-ba1b-b21ce38dc003'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:39.156603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] The token cache is being invalidate because project 89a453fc88ee49548218764a8f837df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:39.157189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:39.158116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:39.159767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1068/2178] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:38 2022] DELETE /identity/v3/projects/89a453fc88ee49548218764a8f837df9 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:39.163848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ddc24-3fbd-448d-a1d5-45ba12dabd04 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:39.174621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ddc24-3fbd-448d-a1d5-45ba12dabd04 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:39.193755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:39.237803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:39.238553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:39.238954 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:39.239236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] PATH_INFO: `/v3/users/088958368fea4a3bb0ec87b9e95e0d5c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:39.240378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=088958368fea4a3bb0ec87b9e95e0d5c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:39.242350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:39.499682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088958368fea4a3bb0ec87b9e95e0d5c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:39.513412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088958368fea4a3bb0ec87b9e95e0d5c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:39.522172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088958368fea4a3bb0ec87b9e95e0d5c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:39.567528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088958368fea4a3bb0ec87b9e95e0d5c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:39.568304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088958368fea4a3bb0ec87b9e95e0d5c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:39.575621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1111/2179] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:39 2022] DELETE /identity/v3/users/088958368fea4a3bb0ec87b9e95e0d5c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:39.842177 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:39.843624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:39.904941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e2ad6-542b-4952-ba1b-b21ce38dc003 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:39.944803 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:39.956321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c87969-e141-4642-a608-00fad0aa00c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:39.956923 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c87969-e141-4642-a608-00fad0aa00c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:39.957194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c87969-e141-4642-a608-00fad0aa00c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:39.957479 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c87969-e141-4642-a608-00fad0aa00c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:39.959208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-49c87969-e141-4642-a608-00fad0aa00c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:39.994295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:39.996501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:40.034683 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c87969-e141-4642-a608-00fad0aa00c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:40.034683 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c87969-e141-4642-a608-00fad0aa00c6 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:40.041392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1069/2180] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:40.087111 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd00b8d-4f6f-47ff-9b56-b7eea7896a3f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:40.092501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:40.093133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:40.093413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:40.094280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] PATH_INFO: `/v3/projects/2c7d976754f64c95806a9ec17ba0ec8f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:40.095513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7d976754f64c95806a9ec17ba0ec8f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:40.097532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:40.194035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:40.197750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7d976754f64c95806a9ec17ba0ec8f', 'request_id': 'req-375dc926-1da4-401d-8ac8-51440bbd5837'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:40.277266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7d976754f64c95806a9ec17ba0ec8f', 'request_id': 'req-375dc926-1da4-401d-8ac8-51440bbd5837'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:40.286184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7d976754f64c95806a9ec17ba0ec8f', 'request_id': 'req-375dc926-1da4-401d-8ac8-51440bbd5837'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:40.296775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] The token cache is being invalidate because project 2c7d976754f64c95806a9ec17ba0ec8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:40.297190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:40.298109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-375dc926-1da4-401d-8ac8-51440bbd5837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:40.299613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1112/2181] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:39 2022] DELETE /identity/v3/projects/2c7d976754f64c95806a9ec17ba0ec8f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:40.304466 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abab038-c75d-4717-a176-74b05a7927fc None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:40.304954 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abab038-c75d-4717-a176-74b05a7927fc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:40.305041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abab038-c75d-4717-a176-74b05a7927fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:40.326170 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3abab038-c75d-4717-a176-74b05a7927fc None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:40.327376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3abab038-c75d-4717-a176-74b05a7927fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:40.411350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1070/2182] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:40.413225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11937c-1617-4c01-a013-a7c9e8874ae9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:40.414355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11937c-1617-4c01-a013-a7c9e8874ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:40.414709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11937c-1617-4c01-a013-a7c9e8874ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:40.436272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cb11937c-1617-4c01-a013-a7c9e8874ae9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:40.438445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cb11937c-1617-4c01-a013-a7c9e8874ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:40.446655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1113/2183] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:40.450770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abab038-c75d-4717-a176-74b05a7927fc None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:40.461649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abab038-c75d-4717-a176-74b05a7927fc None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:40.466192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f5685-1913-4633-9754-cb4e0c93bef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:40.466779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f5685-1913-4633-9754-cb4e0c93bef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:40.467056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f5685-1913-4633-9754-cb4e0c93bef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:40.467326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f5685-1913-4633-9754-cb4e0c93bef5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:40.467965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f5685-1913-4633-9754-cb4e0c93bef5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:33:40.468264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f5685-1913-4633-9754-cb4e0c93bef5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:40.470351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f5685-1913-4633-9754-cb4e0c93bef5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:40.482203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1071/2184] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:33:40.486623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11937c-1617-4c01-a013-a7c9e8874ae9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:40.495977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11937c-1617-4c01-a013-a7c9e8874ae9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:40.501878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a608e86-b170-451f-b48e-6f218f9b1bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:40.502289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a608e86-b170-451f-b48e-6f218f9b1bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:40.502596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a608e86-b170-451f-b48e-6f218f9b1bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:40.502839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a608e86-b170-451f-b48e-6f218f9b1bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:40.503474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a608e86-b170-451f-b48e-6f218f9b1bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:40.505513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a608e86-b170-451f-b48e-6f218f9b1bb3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:40.621214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1114/2185] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:33:40 2022] POST /identity/v3/projects => generated 355 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:40.626033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f5685-1913-4633-9754-cb4e0c93bef5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:40.636151 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f5685-1913-4633-9754-cb4e0c93bef5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:40.694087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed159ed-761d-418d-b305-b8cc3ac39e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:40.694686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed159ed-761d-418d-b305-b8cc3ac39e86 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:40.694959 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed159ed-761d-418d-b305-b8cc3ac39e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:40.695225 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed159ed-761d-418d-b305-b8cc3ac39e86 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:40.695854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed159ed-761d-418d-b305-b8cc3ac39e86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:40.697783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed159ed-761d-418d-b305-b8cc3ac39e86 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:40.809532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1072/2186] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:33:40 2022] POST /identity/v3/users => generated 278 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:40.811108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a608e86-b170-451f-b48e-6f218f9b1bb3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:40.822615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a608e86-b170-451f-b48e-6f218f9b1bb3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:40.827728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12833dfb-9b1b-45a9-b2b7-3ef610be2934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:40.832588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833dfb-9b1b-45a9-b2b7-3ef610be2934 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:40.832588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833dfb-9b1b-45a9-b2b7-3ef610be2934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:40.832588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833dfb-9b1b-45a9-b2b7-3ef610be2934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:40.832588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12833dfb-9b1b-45a9-b2b7-3ef610be2934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:40.832588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12833dfb-9b1b-45a9-b2b7-3ef610be2934 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:40.853106 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1115/2187] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:40.855259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed159ed-761d-418d-b305-b8cc3ac39e86 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:40.865801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed159ed-761d-418d-b305-b8cc3ac39e86 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:40.870694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b0e31-a1f8-4a58-9b19-832a04ec8f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:40.871382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b0e31-a1f8-4a58-9b19-832a04ec8f00 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:40.871660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b0e31-a1f8-4a58-9b19-832a04ec8f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:40.871932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b0e31-a1f8-4a58-9b19-832a04ec8f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:40.872594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b0e31-a1f8-4a58-9b19-832a04ec8f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:40.874198 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b0e31-a1f8-4a58-9b19-832a04ec8f00 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:40.875744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:40.885979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1073/2188] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:40.891338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12833dfb-9b1b-45a9-b2b7-3ef610be2934 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:40.902034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12833dfb-9b1b-45a9-b2b7-3ef610be2934 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:40.907578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b434c-731b-4ee0-a854-9d8572767bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:40.912900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b434c-731b-4ee0-a854-9d8572767bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:40.912900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b434c-731b-4ee0-a854-9d8572767bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:40.912900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b434c-731b-4ee0-a854-9d8572767bd3 admin admin] PATH_INFO: `/v3/projects/b27dc488ebc24f9eb28aef05503a34e8/users/4872ec31feef423da52072f109969725/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:40.923573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b434c-731b-4ee0-a854-9d8572767bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27dc488ebc24f9eb28aef05503a34e8, user_id=4872ec31feef423da52072f109969725, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:40.926266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b434c-731b-4ee0-a854-9d8572767bd3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:41.026821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1116/2189] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:40 2022] PUT /identity/v3/projects/b27dc488ebc24f9eb28aef05503a34e8/users/4872ec31feef423da52072f109969725/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:41.038265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5db9f6-1b07-473f-91c9-3607956d39ce None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:41.038555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5db9f6-1b07-473f-91c9-3607956d39ce None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:41.038833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5db9f6-1b07-473f-91c9-3607956d39ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:41.067676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-bb5db9f6-1b07-473f-91c9-3607956d39ce None None] MFA Rules not processed for user `4872ec31feef423da52072f109969725`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:41.068809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bb5db9f6-1b07-473f-91c9-3607956d39ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:41.134785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1074/2190] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:41.139210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b434c-731b-4ee0-a854-9d8572767bd3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:41.200649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b434c-731b-4ee0-a854-9d8572767bd3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:41.205713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d4220-92c2-48a6-b02a-166db9ac47bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:41.206313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d4220-92c2-48a6-b02a-166db9ac47bd admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:41.206592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d4220-92c2-48a6-b02a-166db9ac47bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:41.206863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d4220-92c2-48a6-b02a-166db9ac47bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:41.207495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d4220-92c2-48a6-b02a-166db9ac47bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:41.210196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d4220-92c2-48a6-b02a-166db9ac47bd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:41.325309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1117/2191] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:33:41 2022] POST /identity/v3/projects => generated 355 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:41.329436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5db9f6-1b07-473f-91c9-3607956d39ce None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:41.340111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5db9f6-1b07-473f-91c9-3607956d39ce None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:41.395325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae1874-ccef-4ff9-b6c4-6974017db125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:41.395922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1874-ccef-4ff9-b6c4-6974017db125 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:41.396197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1874-ccef-4ff9-b6c4-6974017db125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:41.396467 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae1874-ccef-4ff9-b6c4-6974017db125 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:41.397094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae1874-ccef-4ff9-b6c4-6974017db125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:41.399078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae1874-ccef-4ff9-b6c4-6974017db125 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:41.505742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1075/2192] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:33:41 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:33:41.509888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d4220-92c2-48a6-b02a-166db9ac47bd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:41.522585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d4220-92c2-48a6-b02a-166db9ac47bd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:41.527377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301c3e0-bd00-4d66-a421-90eca9411696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:41.527987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301c3e0-bd00-4d66-a421-90eca9411696 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:41.528263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301c3e0-bd00-4d66-a421-90eca9411696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:41.528533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301c3e0-bd00-4d66-a421-90eca9411696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:41.529178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301c3e0-bd00-4d66-a421-90eca9411696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:41.530889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301c3e0-bd00-4d66-a421-90eca9411696 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:41.549568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1118/2193] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:41.553753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae1874-ccef-4ff9-b6c4-6974017db125 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:41.567340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae1874-ccef-4ff9-b6c4-6974017db125 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:41.573200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e444d-b182-45c0-ad82-4a441b4555cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:41.573908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e444d-b182-45c0-ad82-4a441b4555cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:41.574192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e444d-b182-45c0-ad82-4a441b4555cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:41.574465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e444d-b182-45c0-ad82-4a441b4555cf admin admin] PATH_INFO: `/v3/projects/80c88965ebe8430da2f2b4221a618df9/users/ac3c100f2c54467d8ccf374358a3da9e/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:41.595118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e444d-b182-45c0-ad82-4a441b4555cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c88965ebe8430da2f2b4221a618df9, user_id=ac3c100f2c54467d8ccf374358a3da9e, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:41.598055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e444d-b182-45c0-ad82-4a441b4555cf admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:41.708061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1076/2194] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:41 2022] PUT /identity/v3/projects/80c88965ebe8430da2f2b4221a618df9/users/ac3c100f2c54467d8ccf374358a3da9e/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:41.712503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301c3e0-bd00-4d66-a421-90eca9411696 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:41.723558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301c3e0-bd00-4d66-a421-90eca9411696 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:41.769282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:41.786211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e143c-c7c8-49a4-8c05-6f4f5157463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:41.788015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e143c-c7c8-49a4-8c05-6f4f5157463f admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:41.788015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e143c-c7c8-49a4-8c05-6f4f5157463f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:41.788015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e143c-c7c8-49a4-8c05-6f4f5157463f admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:41.788015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e143c-c7c8-49a4-8c05-6f4f5157463f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:41.789568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e143c-c7c8-49a4-8c05-6f4f5157463f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:41.803379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1119/2195] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:41.804814 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e444d-b182-45c0-ad82-4a441b4555cf admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:41.817075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e444d-b182-45c0-ad82-4a441b4555cf admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:41.822475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d86262-a139-4220-ab02-f35e5107939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:41.823214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86262-a139-4220-ab02-f35e5107939b admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:41.823600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86262-a139-4220-ab02-f35e5107939b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:41.824013 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d86262-a139-4220-ab02-f35e5107939b admin admin] PATH_INFO: `/v3/projects/80c88965ebe8430da2f2b4221a618df9/users/ac3c100f2c54467d8ccf374358a3da9e/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:41.826007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d86262-a139-4220-ab02-f35e5107939b admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c88965ebe8430da2f2b4221a618df9, user_id=ac3c100f2c54467d8ccf374358a3da9e, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:41.829167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d86262-a139-4220-ab02-f35e5107939b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:41.840635 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1077/2196] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:41 2022] PUT /identity/v3/projects/80c88965ebe8430da2f2b4221a618df9/users/ac3c100f2c54467d8ccf374358a3da9e/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:41.851166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39328eae-b998-446c-a7b5-b671db6966ef None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:41.851444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39328eae-b998-446c-a7b5-b671db6966ef None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:41.851666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39328eae-b998-446c-a7b5-b671db6966ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:41.881962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-39328eae-b998-446c-a7b5-b671db6966ef None None] MFA Rules not processed for user `ac3c100f2c54467d8ccf374358a3da9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:41.883313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-39328eae-b998-446c-a7b5-b671db6966ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:41.939512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d86262-a139-4220-ab02-f35e5107939b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:41.941234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-12d86262-a139-4220-ab02-f35e5107939b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:41.965669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1120/2197] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:41.988459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:42.010632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d86262-a139-4220-ab02-f35e5107939b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:42.015453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:42.016103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:42.016375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:42.016639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] PATH_INFO: `/v3/users/9731ff85af1c4b5fa8b2167dd5ddc728` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:42.017790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9731ff85af1c4b5fa8b2167dd5ddc728)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:42.019707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:42.169861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39328eae-b998-446c-a7b5-b671db6966ef None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:42.171283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-39328eae-b998-446c-a7b5-b671db6966ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:42.235094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39328eae-b998-446c-a7b5-b671db6966ef None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:42.235738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9731ff85af1c4b5fa8b2167dd5ddc728'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:42.236463 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9731ff85af1c4b5fa8b2167dd5ddc728'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:42.238886 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740c0e8-a954-448a-83fe-1515031f102e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:42.239494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c0e8-a954-448a-83fe-1515031f102e service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:42.239768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c0e8-a954-448a-83fe-1515031f102e service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:42.240034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c0e8-a954-448a-83fe-1515031f102e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:42.241787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7740c0e8-a954-448a-83fe-1515031f102e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:42.253404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740c0e8-a954-448a-83fe-1515031f102e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:42.254877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740c0e8-a954-448a-83fe-1515031f102e service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:42.263957 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1121/2198] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:33:42.318730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9731ff85af1c4b5fa8b2167dd5ddc728'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:42.334215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9731ff85af1c4b5fa8b2167dd5ddc728'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:42.341909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9731ff85af1c4b5fa8b2167dd5ddc728'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:42.348647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1078/2199] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:41 2022] DELETE /identity/v3/users/9731ff85af1c4b5fa8b2167dd5ddc728 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:42.353238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a8872-f710-4708-a328-50b22942c8cf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:42.353547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a8872-f710-4708-a328-50b22942c8cf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:42.353825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a8872-f710-4708-a328-50b22942c8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:42.375070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ea8a8872-f710-4708-a328-50b22942c8cf None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:42.375070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ea8a8872-f710-4708-a328-50b22942c8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:42.446110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1122/2200] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:42.457768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:42.459509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:42.501138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a8872-f710-4708-a328-50b22942c8cf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:42.502657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ea8a8872-f710-4708-a328-50b22942c8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:42.513840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:42.524540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0cc04-0891-4289-b954-47783cf3854a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:42.528187 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f73cf-1296-4117-9e77-198a71ba1be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:42.530530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f73cf-1296-4117-9e77-198a71ba1be4 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:42.530530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f73cf-1296-4117-9e77-198a71ba1be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:42.530530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f73cf-1296-4117-9e77-198a71ba1be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:42.539606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f73cf-1296-4117-9e77-198a71ba1be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:42.541050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f73cf-1296-4117-9e77-198a71ba1be4 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:42.552197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1079/2201] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:42.565229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a8872-f710-4708-a328-50b22942c8cf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:42.569230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f85e65-a055-4e37-97c8-d19a693531b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:42.569985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f85e65-a055-4e37-97c8-d19a693531b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:42.570359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f85e65-a055-4e37-97c8-d19a693531b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:42.570703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f85e65-a055-4e37-97c8-d19a693531b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:42.572634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f2f85e65-a055-4e37-97c8-d19a693531b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:42.637468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f85e65-a055-4e37-97c8-d19a693531b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:42.639040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f85e65-a055-4e37-97c8-d19a693531b4 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:42.649162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1123/2202] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:42.994017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f73cf-1296-4117-9e77-198a71ba1be4 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:42.995685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-903f73cf-1296-4117-9e77-198a71ba1be4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:43.009887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f73cf-1296-4117-9e77-198a71ba1be4 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:43.015139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:43.016201 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:43.016495 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:43.016843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] PATH_INFO: `/v3/projects/0bff778f3b4646c88fad2c4b6ac98a93` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:43.018352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bff778f3b4646c88fad2c4b6ac98a93)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:43.019972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:43.169012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bff778f3b4646c88fad2c4b6ac98a93', 'request_id': 'req-f25acd77-1a02-4522-8878-fa6bab491d98'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:43.172934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:43.255534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bff778f3b4646c88fad2c4b6ac98a93', 'request_id': 'req-f25acd77-1a02-4522-8878-fa6bab491d98'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:43.262973 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bff778f3b4646c88fad2c4b6ac98a93', 'request_id': 'req-f25acd77-1a02-4522-8878-fa6bab491d98'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:43.274454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] The token cache is being invalidate because project 0bff778f3b4646c88fad2c4b6ac98a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:43.274743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:43.275400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f25acd77-1a02-4522-8878-fa6bab491d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:43.276982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1080/2203] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:42 2022] DELETE /identity/v3/projects/0bff778f3b4646c88fad2c4b6ac98a93 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:43.282648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f694-f5ff-499b-a67c-fec797f99da9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:43.283112 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f694-f5ff-499b-a67c-fec797f99da9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:43.283206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f694-f5ff-499b-a67c-fec797f99da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:43.302491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7b03f694-f5ff-499b-a67c-fec797f99da9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:43.303688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7b03f694-f5ff-499b-a67c-fec797f99da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:43.364408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1124/2204] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:43.372253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79536e0a-cb87-4b92-8af0-4a315213fc9c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:43.372745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79536e0a-cb87-4b92-8af0-4a315213fc9c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:43.372838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79536e0a-cb87-4b92-8af0-4a315213fc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:43.393942 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-79536e0a-cb87-4b92-8af0-4a315213fc9c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:43.395185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-79536e0a-cb87-4b92-8af0-4a315213fc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:43.403215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1081/2205] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:43.407554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03f694-f5ff-499b-a67c-fec797f99da9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:43.417088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03f694-f5ff-499b-a67c-fec797f99da9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:43.421774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b6152-e6df-4e17-9a31-0785e0c708fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:43.422364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b6152-e6df-4e17-9a31-0785e0c708fd admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:43.422644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b6152-e6df-4e17-9a31-0785e0c708fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:43.422964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b6152-e6df-4e17-9a31-0785e0c708fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:43.423610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b6152-e6df-4e17-9a31-0785e0c708fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:33:43.423913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b6152-e6df-4e17-9a31-0785e0c708fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:43.425533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b6152-e6df-4e17-9a31-0785e0c708fd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:43.438757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1125/2206] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:33:43.443488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79536e0a-cb87-4b92-8af0-4a315213fc9c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:43.452651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79536e0a-cb87-4b92-8af0-4a315213fc9c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:43.458592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab3cef-8ad9-460a-ae95-d0a9a782c7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:43.458992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab3cef-8ad9-460a-ae95-d0a9a782c7bc admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:43.459268 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab3cef-8ad9-460a-ae95-d0a9a782c7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:43.459537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab3cef-8ad9-460a-ae95-d0a9a782c7bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:43.460170 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab3cef-8ad9-460a-ae95-d0a9a782c7bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:43.462098 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab3cef-8ad9-460a-ae95-d0a9a782c7bc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:43.586902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1082/2207] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:33:43 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:33:43.591326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b6152-e6df-4e17-9a31-0785e0c708fd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:43.601709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b6152-e6df-4e17-9a31-0785e0c708fd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:43.648788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:43.661071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6563de5-8ca3-479a-9a7e-681bd5162a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:43.661838 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6563de5-8ca3-479a-9a7e-681bd5162a21 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:43.662124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6563de5-8ca3-479a-9a7e-681bd5162a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:43.662396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6563de5-8ca3-479a-9a7e-681bd5162a21 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:43.663038 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6563de5-8ca3-479a-9a7e-681bd5162a21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:43.664968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6563de5-8ca3-479a-9a7e-681bd5162a21 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:43.793142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1126/2208] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:33:43 2022] POST /identity/v3/users => generated 291 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:43.797980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab3cef-8ad9-460a-ae95-d0a9a782c7bc admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:43.809282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab3cef-8ad9-460a-ae95-d0a9a782c7bc admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:43.814746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51ed11-cafc-4cd0-9af6-703194db30dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:43.815351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51ed11-cafc-4cd0-9af6-703194db30dd admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:43.815770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51ed11-cafc-4cd0-9af6-703194db30dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:43.816045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51ed11-cafc-4cd0-9af6-703194db30dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:43.816691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51ed11-cafc-4cd0-9af6-703194db30dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:43.818350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51ed11-cafc-4cd0-9af6-703194db30dd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:43.830103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1083/2209] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:43.835248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6563de5-8ca3-479a-9a7e-681bd5162a21 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:43.846078 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6563de5-8ca3-479a-9a7e-681bd5162a21 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:43.850766 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59af643-ccba-4d95-9171-cba1d7de5825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:43.851364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59af643-ccba-4d95-9171-cba1d7de5825 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:43.851649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59af643-ccba-4d95-9171-cba1d7de5825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:43.851944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59af643-ccba-4d95-9171-cba1d7de5825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:43.852589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59af643-ccba-4d95-9171-cba1d7de5825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:43.858157 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59af643-ccba-4d95-9171-cba1d7de5825 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:43.865564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1127/2210] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:43.869665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51ed11-cafc-4cd0-9af6-703194db30dd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:43.880220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51ed11-cafc-4cd0-9af6-703194db30dd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:43.884700 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b384af-0deb-48a0-bf04-b25f2e19e1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:43.885339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b384af-0deb-48a0-bf04-b25f2e19e1b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:43.885652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b384af-0deb-48a0-bf04-b25f2e19e1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:43.886599 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b384af-0deb-48a0-bf04-b25f2e19e1b8 admin admin] PATH_INFO: `/v3/projects/10dd440ea6f54e0497536117ccba4c9e/users/b2c0f79be80341a483f02509d774007f/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:43.901281 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b384af-0deb-48a0-bf04-b25f2e19e1b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10dd440ea6f54e0497536117ccba4c9e, user_id=b2c0f79be80341a483f02509d774007f, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:43.903962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b384af-0deb-48a0-bf04-b25f2e19e1b8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:43.972971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1084/2211] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:43 2022] PUT /identity/v3/projects/10dd440ea6f54e0497536117ccba4c9e/users/b2c0f79be80341a483f02509d774007f/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:43.973826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d7869-4a11-4515-a0ea-e1ea884bd27f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:43.974122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d7869-4a11-4515-a0ea-e1ea884bd27f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:43.974397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d7869-4a11-4515-a0ea-e1ea884bd27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:43.993680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-697d7869-4a11-4515-a0ea-e1ea884bd27f None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:43.995681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-697d7869-4a11-4515-a0ea-e1ea884bd27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:44.058114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1128/2212] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:44.067418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b384af-0deb-48a0-bf04-b25f2e19e1b8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:44.069118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b2b384af-0deb-48a0-bf04-b25f2e19e1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:44.129753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b384af-0deb-48a0-bf04-b25f2e19e1b8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:44.133723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643b958-f928-4045-b95b-c15d1001b8d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:44.134396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643b958-f928-4045-b95b-c15d1001b8d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:44.134593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643b958-f928-4045-b95b-c15d1001b8d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:44.134861 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643b958-f928-4045-b95b-c15d1001b8d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:44.145935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643b958-f928-4045-b95b-c15d1001b8d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:44.147602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643b958-f928-4045-b95b-c15d1001b8d9 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:44.170661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1085/2213] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:46.093325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d7869-4a11-4515-a0ea-e1ea884bd27f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:46.095015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-697d7869-4a11-4515-a0ea-e1ea884bd27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:46.099949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353f03c-79f4-4146-a891-7c456e251e6c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:46.100428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353f03c-79f4-4146-a891-7c456e251e6c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:46.100428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353f03c-79f4-4146-a891-7c456e251e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:46.114311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d7869-4a11-4515-a0ea-e1ea884bd27f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:46.119824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:46.121563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:46.122028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:46.122674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] PATH_INFO: `/v3/users/a52c2a9fc56144d9bfaf4c18566b8096` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:46.124236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52c2a9fc56144d9bfaf4c18566b8096)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:46.127312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:46.141820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4353f03c-79f4-4146-a891-7c456e251e6c None None] MFA Rules not processed for user `b2c0f79be80341a483f02509d774007f`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:46.146341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4353f03c-79f4-4146-a891-7c456e251e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:46.221763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1086/2214] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:46.233051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353f03c-79f4-4146-a891-7c456e251e6c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:46.235440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4353f03c-79f4-4146-a891-7c456e251e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:46.251141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:46.298346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353f03c-79f4-4146-a891-7c456e251e6c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:46.302012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17403387-ec25-4d3e-ab54-ebf94fc9c5e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:46.302613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17403387-ec25-4d3e-ab54-ebf94fc9c5e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:46.302887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17403387-ec25-4d3e-ab54-ebf94fc9c5e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:46.303155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17403387-ec25-4d3e-ab54-ebf94fc9c5e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:46.312130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17403387-ec25-4d3e-ab54-ebf94fc9c5e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:46.313701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17403387-ec25-4d3e-ab54-ebf94fc9c5e3 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:46.323466 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1087/2215] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:33:46.359258 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52c2a9fc56144d9bfaf4c18566b8096'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:46.373287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52c2a9fc56144d9bfaf4c18566b8096'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:46.385349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52c2a9fc56144d9bfaf4c18566b8096'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:46.467408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52c2a9fc56144d9bfaf4c18566b8096'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:46.468814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5e7a808-7e01-45b5-9bf8-0b927db32728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52c2a9fc56144d9bfaf4c18566b8096'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:46.478231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1129/2216] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:46 2022] DELETE /identity/v3/users/a52c2a9fc56144d9bfaf4c18566b8096 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:46.483216 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fc7d5-078a-4710-92fd-f93ce9eb9ad4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:46.483506 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fc7d5-078a-4710-92fd-f93ce9eb9ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:46.484059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fc7d5-078a-4710-92fd-f93ce9eb9ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:46.507269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2b6fc7d5-078a-4710-92fd-f93ce9eb9ad4 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:46.507269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2b6fc7d5-078a-4710-92fd-f93ce9eb9ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:46.538142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e926b-0545-49c8-b0d2-c050d9310144 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:46.538437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e926b-0545-49c8-b0d2-c050d9310144 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:46.538717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e926b-0545-49c8-b0d2-c050d9310144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:46.560273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0c9e926b-0545-49c8-b0d2-c050d9310144 None None] MFA Rules not processed for user `fb50b15e44814c6b82b2a75e3c74de0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:46.561538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0c9e926b-0545-49c8-b0d2-c050d9310144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:46.570632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1088/2217] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:46.580787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fc7d5-078a-4710-92fd-f93ce9eb9ad4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:46.582571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2b6fc7d5-078a-4710-92fd-f93ce9eb9ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:46.601399 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:46.628584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1130/2218] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:46.653544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fc7d5-078a-4710-92fd-f93ce9eb9ad4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:46.657612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5b84f-954e-4d96-be12-380d2c11a633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:46.658213 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5b84f-954e-4d96-be12-380d2c11a633 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:46.658608 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5b84f-954e-4d96-be12-380d2c11a633 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:46.658889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5b84f-954e-4d96-be12-380d2c11a633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:46.668638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5b84f-954e-4d96-be12-380d2c11a633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:46.670098 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5b84f-954e-4d96-be12-380d2c11a633 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:46.680469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1089/2219] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:33:47.346755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e926b-0545-49c8-b0d2-c050d9310144 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:47.348712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0c9e926b-0545-49c8-b0d2-c050d9310144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:47.366550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e926b-0545-49c8-b0d2-c050d9310144 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:47.371489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:47.372126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:47.372404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:47.372675 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] PATH_INFO: `/v3/projects/ff7b630ffa4f429498fe145ce35ca4d3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:47.374014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff7b630ffa4f429498fe145ce35ca4d3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:47.376273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:47.540551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff7b630ffa4f429498fe145ce35ca4d3', 'request_id': 'req-5edb68c4-007a-4324-b3f0-3840ea9c8f11'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:47.612938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff7b630ffa4f429498fe145ce35ca4d3', 'request_id': 'req-5edb68c4-007a-4324-b3f0-3840ea9c8f11'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:47.620167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff7b630ffa4f429498fe145ce35ca4d3', 'request_id': 'req-5edb68c4-007a-4324-b3f0-3840ea9c8f11'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:47.631182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] The token cache is being invalidate because project ff7b630ffa4f429498fe145ce35ca4d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:47.631496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:47.632117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:47.633657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1131/2220] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:47 2022] DELETE /identity/v3/projects/ff7b630ffa4f429498fe145ce35ca4d3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:47.638213 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5b84f-954e-4d96-be12-380d2c11a633 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:47.650524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5b84f-954e-4d96-be12-380d2c11a633 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:47.707397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:47.715943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:47.715943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:47.715943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:47.715943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] PATH_INFO: `/v3/users/2660f32b04194a14a82ef2571e73d9cd` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:47.716344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2660f32b04194a14a82ef2571e73d9cd)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:47.717611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:47.942296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2660f32b04194a14a82ef2571e73d9cd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:47.942947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2660f32b04194a14a82ef2571e73d9cd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:47.995745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2660f32b04194a14a82ef2571e73d9cd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:48.009440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2660f32b04194a14a82ef2571e73d9cd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:48.017020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-16db931d-4a6d-4ab0-984f-10868b6d1482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2660f32b04194a14a82ef2571e73d9cd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:48.024691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1090/2221] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:47 2022] DELETE /identity/v3/users/2660f32b04194a14a82ef2571e73d9cd => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:48.461721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:48.463185 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:48.477690 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb68c4-007a-4324-b3f0-3840ea9c8f11 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:48.538144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:48.538776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:48.539050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:48.539344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] PATH_INFO: `/v3/projects/397b6b3a855447ce8acbe5ac42ec4ddd` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:48.540447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=397b6b3a855447ce8acbe5ac42ec4ddd)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:48.542417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:48.667062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '397b6b3a855447ce8acbe5ac42ec4ddd', 'request_id': 'req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:48.748819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '397b6b3a855447ce8acbe5ac42ec4ddd', 'request_id': 'req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:48.760214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '397b6b3a855447ce8acbe5ac42ec4ddd', 'request_id': 'req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:48.770178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:48.775465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] The token cache is being invalidate because project 397b6b3a855447ce8acbe5ac42ec4ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:48.775810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:48.776639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0e15dab4-e51c-46d2-b4f8-77b3a3a03545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:48.778380 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1132/2222] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:48 2022] DELETE /identity/v3/projects/397b6b3a855447ce8acbe5ac42ec4ddd => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:48.783907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00767953-f029-4be3-b892-139956e3b015 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:48.784389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00767953-f029-4be3-b892-139956e3b015 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:48.784481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00767953-f029-4be3-b892-139956e3b015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:48.806830 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-00767953-f029-4be3-b892-139956e3b015 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:48.808085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-00767953-f029-4be3-b892-139956e3b015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:48.874747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1091/2223] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:48.882944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea4116-1f0b-4639-9d21-89fce6bf4bbd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:48.883239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea4116-1f0b-4639-9d21-89fce6bf4bbd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:48.883553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea4116-1f0b-4639-9d21-89fce6bf4bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:48.904268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-31ea4116-1f0b-4639-9d21-89fce6bf4bbd None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:48.906088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-31ea4116-1f0b-4639-9d21-89fce6bf4bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:48.914411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1133/2224] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:48.919069 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00767953-f029-4be3-b892-139956e3b015 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:48.932873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00767953-f029-4be3-b892-139956e3b015 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:48.938110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e0eca-afe1-40cf-b95a-4ca29e70d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:48.938801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e0eca-afe1-40cf-b95a-4ca29e70d453 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:48.939219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e0eca-afe1-40cf-b95a-4ca29e70d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:48.939557 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e0eca-afe1-40cf-b95a-4ca29e70d453 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:48.940287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e0eca-afe1-40cf-b95a-4ca29e70d453 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:33:48.940669 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e0eca-afe1-40cf-b95a-4ca29e70d453 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:48.942412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e0eca-afe1-40cf-b95a-4ca29e70d453 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:48.957150 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1092/2225] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:33:48.974549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea4116-1f0b-4639-9d21-89fce6bf4bbd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:48.986455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea4116-1f0b-4639-9d21-89fce6bf4bbd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:48.992374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48bbb4-9936-46a5-94cb-00408ee1e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:48.992374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48bbb4-9936-46a5-94cb-00408ee1e09c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:48.992374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48bbb4-9936-46a5-94cb-00408ee1e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:49.001697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48bbb4-9936-46a5-94cb-00408ee1e09c admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:49.001697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48bbb4-9936-46a5-94cb-00408ee1e09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:49.001697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48bbb4-9936-46a5-94cb-00408ee1e09c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:49.140733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1134/2226] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:33:48 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:49.142198 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e0eca-afe1-40cf-b95a-4ca29e70d453 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:49.154144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e0eca-afe1-40cf-b95a-4ca29e70d453 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:49.222974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc96c91-18e2-46b0-8f96-84d46bb96529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:49.223581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc96c91-18e2-46b0-8f96-84d46bb96529 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:49.223855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc96c91-18e2-46b0-8f96-84d46bb96529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:49.224120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc96c91-18e2-46b0-8f96-84d46bb96529 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:49.224741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc96c91-18e2-46b0-8f96-84d46bb96529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:49.226698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc96c91-18e2-46b0-8f96-84d46bb96529 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:49.242567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:49.360792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1093/2227] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:33:49 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:49.365112 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48bbb4-9936-46a5-94cb-00408ee1e09c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:49.383596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48bbb4-9936-46a5-94cb-00408ee1e09c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:49.390947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03468d1e-dfb6-4af0-89c5-8829a569b4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:49.391779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468d1e-dfb6-4af0-89c5-8829a569b4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:49.392213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468d1e-dfb6-4af0-89c5-8829a569b4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:49.392963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468d1e-dfb6-4af0-89c5-8829a569b4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:49.393398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03468d1e-dfb6-4af0-89c5-8829a569b4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:49.395641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03468d1e-dfb6-4af0-89c5-8829a569b4e9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:49.413857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc96c91-18e2-46b0-8f96-84d46bb96529 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:49.422604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1135/2228] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:49.431415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc96c91-18e2-46b0-8f96-84d46bb96529 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:49.436237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e996d4-432b-49ec-9350-c41749d8452f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:49.436888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e996d4-432b-49ec-9350-c41749d8452f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:49.437481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e996d4-432b-49ec-9350-c41749d8452f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:49.438059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e945f-de75-4c5b-9f5a-c9d8d4f614d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:49.438666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e945f-de75-4c5b-9f5a-c9d8d4f614d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:49.438947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e945f-de75-4c5b-9f5a-c9d8d4f614d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:49.439219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e945f-de75-4c5b-9f5a-c9d8d4f614d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:49.439927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e945f-de75-4c5b-9f5a-c9d8d4f614d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:49.441747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e945f-de75-4c5b-9f5a-c9d8d4f614d0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:49.455442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1094/2229] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:49.459733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e945f-de75-4c5b-9f5a-c9d8d4f614d0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:49.461249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b3e996d4-432b-49ec-9350-c41749d8452f None None] MFA Rules not processed for user `b2c0f79be80341a483f02509d774007f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:49.462573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b3e996d4-432b-49ec-9350-c41749d8452f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:49.471447 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e945f-de75-4c5b-9f5a-c9d8d4f614d0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:49.477210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17556898-595f-403a-b8ff-8a43a91c7771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:49.477969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17556898-595f-403a-b8ff-8a43a91c7771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:49.478174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17556898-595f-403a-b8ff-8a43a91c7771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:49.478568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17556898-595f-403a-b8ff-8a43a91c7771 admin admin] PATH_INFO: `/v3/projects/a694d6125e9c4dfe97705f53a93f47bd/users/2b1947c8d8ec448e832cc573d96dac6b/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:49.496819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17556898-595f-403a-b8ff-8a43a91c7771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a694d6125e9c4dfe97705f53a93f47bd, user_id=2b1947c8d8ec448e832cc573d96dac6b, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:49.501413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17556898-595f-403a-b8ff-8a43a91c7771 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:49.523575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1136/2230] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:49.585935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1095/2231] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:49 2022] PUT /identity/v3/projects/a694d6125e9c4dfe97705f53a93f47bd/users/2b1947c8d8ec448e832cc573d96dac6b/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:49.598049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf4452-6891-4674-b22a-afa7259955d5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:49.598350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf4452-6891-4674-b22a-afa7259955d5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:49.598619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf4452-6891-4674-b22a-afa7259955d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:49.628250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fabf4452-6891-4674-b22a-afa7259955d5 None None] MFA Rules not processed for user `2b1947c8d8ec448e832cc573d96dac6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:49.629821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fabf4452-6891-4674-b22a-afa7259955d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:49.703461 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1137/2232] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:49.723676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17556898-595f-403a-b8ff-8a43a91c7771 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:49.725722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-17556898-595f-403a-b8ff-8a43a91c7771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:49.793898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17556898-595f-403a-b8ff-8a43a91c7771 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:49.798482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec8c49-ddc8-4a8e-8956-9efa49e56b7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:49.799094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec8c49-ddc8-4a8e-8956-9efa49e56b7a service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:49.799392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec8c49-ddc8-4a8e-8956-9efa49e56b7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:49.799644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec8c49-ddc8-4a8e-8956-9efa49e56b7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:49.808803 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec8c49-ddc8-4a8e-8956-9efa49e56b7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:49.810280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec8c49-ddc8-4a8e-8956-9efa49e56b7a service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:49.831314 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1096/2233] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:49.841944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf4452-6891-4674-b22a-afa7259955d5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:49.905583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:49.917396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf4452-6891-4674-b22a-afa7259955d5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:49.923184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:49.923819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:49.924118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:49.924544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] PATH_INFO: `/v3/users/2b1947c8d8ec448e832cc573d96dac6b` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:49.925584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1947c8d8ec448e832cc573d96dac6b)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:49.927513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:50.145516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1947c8d8ec448e832cc573d96dac6b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:50.160046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1947c8d8ec448e832cc573d96dac6b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:50.167324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1947c8d8ec448e832cc573d96dac6b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:50.232454 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1947c8d8ec448e832cc573d96dac6b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:50.233158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1947c8d8ec448e832cc573d96dac6b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:50.241177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1138/2234] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:49 2022] DELETE /identity/v3/users/2b1947c8d8ec448e832cc573d96dac6b => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:50.245381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f17fc-e66c-40ac-a0d3-ead0170253e5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:50.245683 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f17fc-e66c-40ac-a0d3-ead0170253e5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:50.245963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f17fc-e66c-40ac-a0d3-ead0170253e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:50.270361 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c65f17fc-e66c-40ac-a0d3-ead0170253e5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:50.273130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c65f17fc-e66c-40ac-a0d3-ead0170253e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:50.333645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1097/2235] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:50.344516 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:50.346222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:50.426733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4f44a-d070-47a9-a013-40cadb154d5b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:50.431104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd69db-4252-487b-9d7f-6d61628e59b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:50.431758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd69db-4252-487b-9d7f-6d61628e59b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:50.432040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd69db-4252-487b-9d7f-6d61628e59b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:50.432308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd69db-4252-487b-9d7f-6d61628e59b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:50.446517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd69db-4252-487b-9d7f-6d61628e59b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:50.447940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd69db-4252-487b-9d7f-6d61628e59b1 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:50.459291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1139/2236] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:50.988198 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f17fc-e66c-40ac-a0d3-ead0170253e5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:50.989825 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c65f17fc-e66c-40ac-a0d3-ead0170253e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:51.005928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f17fc-e66c-40ac-a0d3-ead0170253e5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:51.023807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:51.025087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:51.025496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:51.026033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] PATH_INFO: `/v3/projects/a694d6125e9c4dfe97705f53a93f47bd` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:51.027997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a694d6125e9c4dfe97705f53a93f47bd)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:51.030856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:51.148920 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a694d6125e9c4dfe97705f53a93f47bd', 'request_id': 'req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:51.264210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a694d6125e9c4dfe97705f53a93f47bd', 'request_id': 'req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:51.265751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:51.273733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a694d6125e9c4dfe97705f53a93f47bd', 'request_id': 'req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:51.285765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] The token cache is being invalidate because project a694d6125e9c4dfe97705f53a93f47bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:51.286302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:51.287201 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9fe9e309-0ed7-4462-b58f-8c4d4eda053c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:51.289330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1098/2237] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:50 2022] DELETE /identity/v3/projects/a694d6125e9c4dfe97705f53a93f47bd => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:51.294230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d44e3-9af1-438e-aff8-7e9d9b522441 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:51.297755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d44e3-9af1-438e-aff8-7e9d9b522441 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:51.298557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d44e3-9af1-438e-aff8-7e9d9b522441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:51.322805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-061d44e3-9af1-438e-aff8-7e9d9b522441 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:51.324274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-061d44e3-9af1-438e-aff8-7e9d9b522441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:51.394941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1140/2238] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:51.399925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2d11-d6d4-4a30-854a-a3d935737b57 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:51.400212 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2d11-d6d4-4a30-854a-a3d935737b57 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:51.400481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2d11-d6d4-4a30-854a-a3d935737b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:51.421719 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-90aa2d11-d6d4-4a30-854a-a3d935737b57 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:51.426982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-90aa2d11-d6d4-4a30-854a-a3d935737b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:51.431491 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1099/2239] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:51.435742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d44e3-9af1-438e-aff8-7e9d9b522441 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:51.446519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d44e3-9af1-438e-aff8-7e9d9b522441 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:51.451183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed4023-eabd-40dd-8836-e6e5926a10e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:51.451786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed4023-eabd-40dd-8836-e6e5926a10e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:51.455634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed4023-eabd-40dd-8836-e6e5926a10e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:51.455634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed4023-eabd-40dd-8836-e6e5926a10e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:51.455634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed4023-eabd-40dd-8836-e6e5926a10e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:33:51.455634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed4023-eabd-40dd-8836-e6e5926a10e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:51.455634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed4023-eabd-40dd-8836-e6e5926a10e1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:51.467332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1141/2240] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:33:51.475130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa2d11-d6d4-4a30-854a-a3d935737b57 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:51.486381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa2d11-d6d4-4a30-854a-a3d935737b57 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:51.491243 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11185ee0-c479-4a82-9673-454b50375aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:51.491841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11185ee0-c479-4a82-9673-454b50375aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:51.492114 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11185ee0-c479-4a82-9673-454b50375aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:51.492378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11185ee0-c479-4a82-9673-454b50375aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:51.493023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11185ee0-c479-4a82-9673-454b50375aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:51.494946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11185ee0-c479-4a82-9673-454b50375aa3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:51.597276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1100/2241] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:33:51 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:51.598168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed4023-eabd-40dd-8836-e6e5926a10e1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:51.605167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:51.612192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed4023-eabd-40dd-8836-e6e5926a10e1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:51.676466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f760f47-2ed7-4c7d-8351-b57a9d9be307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:51.676953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f760f47-2ed7-4c7d-8351-b57a9d9be307 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:51.677131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f760f47-2ed7-4c7d-8351-b57a9d9be307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:51.677381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f760f47-2ed7-4c7d-8351-b57a9d9be307 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:51.678436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f760f47-2ed7-4c7d-8351-b57a9d9be307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:51.680368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f760f47-2ed7-4c7d-8351-b57a9d9be307 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:51.774488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1142/2242] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:33:51 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:51.779315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11185ee0-c479-4a82-9673-454b50375aa3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:51.792415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11185ee0-c479-4a82-9673-454b50375aa3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:51.799098 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee5a3a-cae8-4b5f-8b66-54d7db448ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:51.799991 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee5a3a-cae8-4b5f-8b66-54d7db448ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:51.800483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee5a3a-cae8-4b5f-8b66-54d7db448ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:51.800907 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee5a3a-cae8-4b5f-8b66-54d7db448ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:51.801812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee5a3a-cae8-4b5f-8b66-54d7db448ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:51.803833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee5a3a-cae8-4b5f-8b66-54d7db448ff9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:51.818427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1101/2243] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:33:51.822602 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f760f47-2ed7-4c7d-8351-b57a9d9be307 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:51.836208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f760f47-2ed7-4c7d-8351-b57a9d9be307 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:51.842293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70cb4a-3ee3-4a4c-822d-af5a2cf007b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:51.842987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70cb4a-3ee3-4a4c-822d-af5a2cf007b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:51.843479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70cb4a-3ee3-4a4c-822d-af5a2cf007b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:51.843758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70cb4a-3ee3-4a4c-822d-af5a2cf007b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:51.844658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70cb4a-3ee3-4a4c-822d-af5a2cf007b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:51.846370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70cb4a-3ee3-4a4c-822d-af5a2cf007b1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:51.859301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1143/2244] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:51.865587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee5a3a-cae8-4b5f-8b66-54d7db448ff9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:51.877214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee5a3a-cae8-4b5f-8b66-54d7db448ff9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:51.882670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4447e7-b5bd-44a8-a7a2-dd7434039175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:51.883339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4447e7-b5bd-44a8-a7a2-dd7434039175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:51.883618 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4447e7-b5bd-44a8-a7a2-dd7434039175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:51.883892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4447e7-b5bd-44a8-a7a2-dd7434039175 admin admin] PATH_INFO: `/v3/projects/92558739fcb847b9b599840200e0130e/users/c9866719736c48e28f038539bba97dfd/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:51.903043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4447e7-b5bd-44a8-a7a2-dd7434039175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92558739fcb847b9b599840200e0130e, user_id=c9866719736c48e28f038539bba97dfd, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:51.905876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4447e7-b5bd-44a8-a7a2-dd7434039175 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:52.013134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1102/2245] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:51 2022] PUT /identity/v3/projects/92558739fcb847b9b599840200e0130e/users/c9866719736c48e28f038539bba97dfd/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:52.023335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4d91-10cd-414c-9e35-f3edd15dfc29 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:52.023790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4d91-10cd-414c-9e35-f3edd15dfc29 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:52.024101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4d91-10cd-414c-9e35-f3edd15dfc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:52.052281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8b5a4d91-10cd-414c-9e35-f3edd15dfc29 None None] MFA Rules not processed for user `c9866719736c48e28f038539bba97dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:52.053664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8b5a4d91-10cd-414c-9e35-f3edd15dfc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:52.086581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4447e7-b5bd-44a8-a7a2-dd7434039175 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:52.089333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cd4447e7-b5bd-44a8-a7a2-dd7434039175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:52.134897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1144/2246] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:52.142493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4d91-10cd-414c-9e35-f3edd15dfc29 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:52.181908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4447e7-b5bd-44a8-a7a2-dd7434039175 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:52.190742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fb3a1-1d13-4a49-8b15-66753bada18e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:52.191455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb3a1-1d13-4a49-8b15-66753bada18e service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:52.191740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb3a1-1d13-4a49-8b15-66753bada18e service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:52.192008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb3a1-1d13-4a49-8b15-66753bada18e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:52.194061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-934fb3a1-1d13-4a49-8b15-66753bada18e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:52.210868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fb3a1-1d13-4a49-8b15-66753bada18e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:52.212383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fb3a1-1d13-4a49-8b15-66753bada18e service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:52.228652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1103/2247] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:33:52.243902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4d91-10cd-414c-9e35-f3edd15dfc29 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:52.249351 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3bcad-c4ae-4733-a265-3443645e8520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:52.250210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3bcad-c4ae-4733-a265-3443645e8520 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:52.250494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3bcad-c4ae-4733-a265-3443645e8520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:52.254800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3bcad-c4ae-4733-a265-3443645e8520 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:52.254800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3bcad-c4ae-4733-a265-3443645e8520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:52.254800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3bcad-c4ae-4733-a265-3443645e8520 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:52.395266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1145/2248] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:33:52 2022] POST /identity/v3/projects => generated 375 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:33:52.396559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fb3a1-1d13-4a49-8b15-66753bada18e service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:52.407275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fb3a1-1d13-4a49-8b15-66753bada18e service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:52.414282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:52.482877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3422c6-e3f7-4ced-ad2e-a9f202403259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:52.483737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3422c6-e3f7-4ced-ad2e-a9f202403259 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:52.484086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3422c6-e3f7-4ced-ad2e-a9f202403259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:52.484502 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3422c6-e3f7-4ced-ad2e-a9f202403259 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:52.485142 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3422c6-e3f7-4ced-ad2e-a9f202403259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:52.487118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3422c6-e3f7-4ced-ad2e-a9f202403259 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:52.612451 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:52.623006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1104/2249] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:33:52 2022] POST /identity/v3/users => generated 294 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:33:52.624703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3bcad-c4ae-4733-a265-3443645e8520 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:52.637720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3bcad-c4ae-4733-a265-3443645e8520 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:52.645915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7f649-b6d3-4f2c-9a89-845d62aa0c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:52.646780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7f649-b6d3-4f2c-9a89-845d62aa0c07 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:52.647158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7f649-b6d3-4f2c-9a89-845d62aa0c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:52.647604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7f649-b6d3-4f2c-9a89-845d62aa0c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:52.648359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7f649-b6d3-4f2c-9a89-845d62aa0c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:52.650145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7f649-b6d3-4f2c-9a89-845d62aa0c07 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:52.665604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1146/2250] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:52.669262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3422c6-e3f7-4ced-ad2e-a9f202403259 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:52.691226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3422c6-e3f7-4ced-ad2e-a9f202403259 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:52.703343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440af59a-473d-4274-9ad5-ecddc11433da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:52.704104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440af59a-473d-4274-9ad5-ecddc11433da admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:52.704457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440af59a-473d-4274-9ad5-ecddc11433da admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:52.704749 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440af59a-473d-4274-9ad5-ecddc11433da admin admin] PATH_INFO: `/v3/projects/3afdd8c0929246bb9003f5de2c99903f/users/da2b3df541864845ba904636f9e7a587/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:52.733431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440af59a-473d-4274-9ad5-ecddc11433da admin admin] RBAC: Authorizing `identity:create_grant(project_id=3afdd8c0929246bb9003f5de2c99903f, user_id=da2b3df541864845ba904636f9e7a587, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:52.736339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440af59a-473d-4274-9ad5-ecddc11433da admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:52.838348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1105/2251] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:52 2022] PUT /identity/v3/projects/3afdd8c0929246bb9003f5de2c99903f/users/da2b3df541864845ba904636f9e7a587/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:52.842468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7f649-b6d3-4f2c-9a89-845d62aa0c07 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:52.854482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7f649-b6d3-4f2c-9a89-845d62aa0c07 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:52.918016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611ed92-7739-431a-9557-f4a4aeb92f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:52.918540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611ed92-7739-431a-9557-f4a4aeb92f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:52.918825 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611ed92-7739-431a-9557-f4a4aeb92f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:52.919096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611ed92-7739-431a-9557-f4a4aeb92f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:52.919776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611ed92-7739-431a-9557-f4a4aeb92f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:52.921482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611ed92-7739-431a-9557-f4a4aeb92f01 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:52.932945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1147/2252] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:33:52.937288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440af59a-473d-4274-9ad5-ecddc11433da admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:52.947666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440af59a-473d-4274-9ad5-ecddc11433da admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:52.952932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8452104-5f16-468a-ba39-6b3690483015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:52.953640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8452104-5f16-468a-ba39-6b3690483015 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:52.953918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8452104-5f16-468a-ba39-6b3690483015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:52.965632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8452104-5f16-468a-ba39-6b3690483015 admin admin] PATH_INFO: `/v3/projects/3afdd8c0929246bb9003f5de2c99903f/users/da2b3df541864845ba904636f9e7a587/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:52.965632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8452104-5f16-468a-ba39-6b3690483015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3afdd8c0929246bb9003f5de2c99903f, user_id=da2b3df541864845ba904636f9e7a587, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:52.965632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8452104-5f16-468a-ba39-6b3690483015 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:52.971259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1106/2253] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:33:52 2022] PUT /identity/v3/projects/3afdd8c0929246bb9003f5de2c99903f/users/da2b3df541864845ba904636f9e7a587/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:33:52.982877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc94d69-5a41-4d67-8408-b08e962eb36d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:52.983161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc94d69-5a41-4d67-8408-b08e962eb36d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:52.983505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc94d69-5a41-4d67-8408-b08e962eb36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:53.011821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2dc94d69-5a41-4d67-8408-b08e962eb36d None None] MFA Rules not processed for user `da2b3df541864845ba904636f9e7a587`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:53.013073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2dc94d69-5a41-4d67-8408-b08e962eb36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:53.085993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1148/2254] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:53.097662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8452104-5f16-468a-ba39-6b3690483015 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:53.099484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b8452104-5f16-468a-ba39-6b3690483015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:53.160782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8452104-5f16-468a-ba39-6b3690483015 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:53.164498 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e8757-7088-43e1-a31b-78b9bfe20766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:53.165110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e8757-7088-43e1-a31b-78b9bfe20766 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:53.165389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e8757-7088-43e1-a31b-78b9bfe20766 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:53.165694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e8757-7088-43e1-a31b-78b9bfe20766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:53.226694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e8757-7088-43e1-a31b-78b9bfe20766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:53.228358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e8757-7088-43e1-a31b-78b9bfe20766 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:53.239271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1107/2255] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:57.527663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc94d69-5a41-4d67-8408-b08e962eb36d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:57.609321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc94d69-5a41-4d67-8408-b08e962eb36d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:57.617853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:57.618625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:57.618993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:57.619318 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] PATH_INFO: `/v3/users/c9866719736c48e28f038539bba97dfd` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:57.620654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9866719736c48e28f038539bba97dfd)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:57.622835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:57.637540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:57.862806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9866719736c48e28f038539bba97dfd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:57.877063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9866719736c48e28f038539bba97dfd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:57.884395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9866719736c48e28f038539bba97dfd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:57.978387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9866719736c48e28f038539bba97dfd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:57.979044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9866719736c48e28f038539bba97dfd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:57.986675 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1149/2256] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:57 2022] DELETE /identity/v3/users/c9866719736c48e28f038539bba97dfd => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:57.991175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7d987-abdd-49b3-8144-c1eed32ede72 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:57.991653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7d987-abdd-49b3-8144-c1eed32ede72 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:57.991747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7d987-abdd-49b3-8144-c1eed32ede72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:58.012374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-42f7d987-abdd-49b3-8144-c1eed32ede72 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:58.013734 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-42f7d987-abdd-49b3-8144-c1eed32ede72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:58.076459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1108/2257] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:58.086339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:58.087778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:58.146514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078ef7d-93b1-4c80-98a1-4da0274b04bf admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:58.150299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e91543-806d-477e-96bd-ec78f8e5b23a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:58.150919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e91543-806d-477e-96bd-ec78f8e5b23a service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:58.151231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e91543-806d-477e-96bd-ec78f8e5b23a service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:58.151511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e91543-806d-477e-96bd-ec78f8e5b23a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:58.161839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e91543-806d-477e-96bd-ec78f8e5b23a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:58.163267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e91543-806d-477e-96bd-ec78f8e5b23a service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:58.172977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1150/2258] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:33:58.587266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7d987-abdd-49b3-8144-c1eed32ede72 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:58.588704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-42f7d987-abdd-49b3-8144-c1eed32ede72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:58.601955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7d987-abdd-49b3-8144-c1eed32ede72 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:58.608572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:58.609207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:58.609511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:58.609783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] PATH_INFO: `/v3/projects/92558739fcb847b9b599840200e0130e` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:58.610931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=92558739fcb847b9b599840200e0130e)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:58.612866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:58.625780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:58.719503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92558739fcb847b9b599840200e0130e', 'request_id': 'req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:58.827438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92558739fcb847b9b599840200e0130e', 'request_id': 'req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:58.835595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92558739fcb847b9b599840200e0130e', 'request_id': 'req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:58.848261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] The token cache is being invalidate because project 92558739fcb847b9b599840200e0130e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:33:58.848465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:58.849135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:58.850679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1109/2259] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:58 2022] DELETE /identity/v3/projects/92558739fcb847b9b599840200e0130e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:58.854698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e91543-806d-477e-96bd-ec78f8e5b23a service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:58.865758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e91543-806d-477e-96bd-ec78f8e5b23a service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:58.932615 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:58.933276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:58.933625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:58.933892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] PATH_INFO: `/v3/users/da2b3df541864845ba904636f9e7a587` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:58.935111 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da2b3df541864845ba904636f9e7a587)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:58.937253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:59.121856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:59.135428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da2b3df541864845ba904636f9e7a587'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:59.147930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da2b3df541864845ba904636f9e7a587'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:59.154764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da2b3df541864845ba904636f9e7a587'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:59.225442 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da2b3df541864845ba904636f9e7a587'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:59.226089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3d2ed6d6-3e4a-41db-8a76-7c2d3ad35d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da2b3df541864845ba904636f9e7a587'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:33:59.232068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1151/2260] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:33:58 2022] DELETE /identity/v3/users/da2b3df541864845ba904636f9e7a587 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:33:59.665733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:59.667231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:59.682491 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4a11-615d-4b71-8a2f-a6f71de94b8a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:59.750045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:59.750834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:59.751256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:59.751563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] PATH_INFO: `/v3/projects/3afdd8c0929246bb9003f5de2c99903f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:59.752681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3afdd8c0929246bb9003f5de2c99903f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:59.754652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:59.963675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3afdd8c0929246bb9003f5de2c99903f', 'request_id': 'req-a5a2142c-3cf5-452c-b5cd-12a91c84370e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:00.046241 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3afdd8c0929246bb9003f5de2c99903f', 'request_id': 'req-a5a2142c-3cf5-452c-b5cd-12a91c84370e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:00.064259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3afdd8c0929246bb9003f5de2c99903f', 'request_id': 'req-a5a2142c-3cf5-452c-b5cd-12a91c84370e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:00.082200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] The token cache is being invalidate because project 3afdd8c0929246bb9003f5de2c99903f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:00.082488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:00.083157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a5a2142c-3cf5-452c-b5cd-12a91c84370e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:00.084692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1110/2261] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:33:59 2022] DELETE /identity/v3/projects/3afdd8c0929246bb9003f5de2c99903f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:00.089888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223aca9-d1d0-4ac2-982b-006f12411d69 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:00.090369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223aca9-d1d0-4ac2-982b-006f12411d69 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:00.090465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223aca9-d1d0-4ac2-982b-006f12411d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:00.113746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4223aca9-d1d0-4ac2-982b-006f12411d69 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:00.114975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4223aca9-d1d0-4ac2-982b-006f12411d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:00.187151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1152/2262] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:00.194987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fcb9f-aef2-4092-b22a-6a6ec23c71b9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:00.195266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fcb9f-aef2-4092-b22a-6a6ec23c71b9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:00.195584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fcb9f-aef2-4092-b22a-6a6ec23c71b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:00.218212 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3d8fcb9f-aef2-4092-b22a-6a6ec23c71b9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:00.219237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3d8fcb9f-aef2-4092-b22a-6a6ec23c71b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:00.231185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1111/2263] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:00.236411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223aca9-d1d0-4ac2-982b-006f12411d69 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:00.248824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223aca9-d1d0-4ac2-982b-006f12411d69 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:00.254027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe90b9f-3e0c-449b-94ea-431825bd116c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:00.254673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe90b9f-3e0c-449b-94ea-431825bd116c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:00.254953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe90b9f-3e0c-449b-94ea-431825bd116c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:00.255223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe90b9f-3e0c-449b-94ea-431825bd116c admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:00.255930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe90b9f-3e0c-449b-94ea-431825bd116c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:34:00.256234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe90b9f-3e0c-449b-94ea-431825bd116c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:00.258004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe90b9f-3e0c-449b-94ea-431825bd116c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:00.271676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1153/2264] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:34:00.276246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fcb9f-aef2-4092-b22a-6a6ec23c71b9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:00.288125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fcb9f-aef2-4092-b22a-6a6ec23c71b9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:00.293660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad1880-4c05-4b4a-921f-cefecbb5c53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:00.294300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad1880-4c05-4b4a-921f-cefecbb5c53d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:00.294548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad1880-4c05-4b4a-921f-cefecbb5c53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:00.294872 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad1880-4c05-4b4a-921f-cefecbb5c53d admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:00.295728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad1880-4c05-4b4a-921f-cefecbb5c53d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:00.297929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad1880-4c05-4b4a-921f-cefecbb5c53d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:00.513444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1112/2265] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:34:00 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:34:00.518228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe90b9f-3e0c-449b-94ea-431825bd116c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:00.528361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe90b9f-3e0c-449b-94ea-431825bd116c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:00.585849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd7659-1b8d-46e1-adbc-8d2e5b2a79cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:00.586457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd7659-1b8d-46e1-adbc-8d2e5b2a79cf admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:00.586737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd7659-1b8d-46e1-adbc-8d2e5b2a79cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:00.587008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd7659-1b8d-46e1-adbc-8d2e5b2a79cf admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:00.587638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd7659-1b8d-46e1-adbc-8d2e5b2a79cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:00.589578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd7659-1b8d-46e1-adbc-8d2e5b2a79cf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:00.657647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1154/2266] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:34:00 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:00.661868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad1880-4c05-4b4a-921f-cefecbb5c53d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:00.667477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:00.673041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad1880-4c05-4b4a-921f-cefecbb5c53d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:00.679994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9eb42-536b-4cfb-b778-80ba1809f63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:00.679994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9eb42-536b-4cfb-b778-80ba1809f63a admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:00.679994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9eb42-536b-4cfb-b778-80ba1809f63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:00.679994 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9eb42-536b-4cfb-b778-80ba1809f63a admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:00.680417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9eb42-536b-4cfb-b778-80ba1809f63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:00.681855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9eb42-536b-4cfb-b778-80ba1809f63a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:00.695458 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1113/2267] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:00.696668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd7659-1b8d-46e1-adbc-8d2e5b2a79cf admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:00.707608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd7659-1b8d-46e1-adbc-8d2e5b2a79cf admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:00.717487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6839a3e-862c-49b3-bda4-8db1ef78bc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:00.717487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6839a3e-862c-49b3-bda4-8db1ef78bc93 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:00.717945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6839a3e-862c-49b3-bda4-8db1ef78bc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:00.717945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6839a3e-862c-49b3-bda4-8db1ef78bc93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:00.718565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6839a3e-862c-49b3-bda4-8db1ef78bc93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:00.720166 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6839a3e-862c-49b3-bda4-8db1ef78bc93 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:00.732190 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1155/2268] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:00.736390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9eb42-536b-4cfb-b778-80ba1809f63a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:00.750529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9eb42-536b-4cfb-b778-80ba1809f63a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:00.756026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4569a2-68db-48f7-83a5-a76673870246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:00.756761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4569a2-68db-48f7-83a5-a76673870246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:00.757029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4569a2-68db-48f7-83a5-a76673870246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:00.757300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4569a2-68db-48f7-83a5-a76673870246 admin admin] PATH_INFO: `/v3/projects/69a815f490d04c19af4646d7ed0f6e1f/users/8a8e1dd7b9354eb6b1ec882efd443dd6/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:00.774703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4569a2-68db-48f7-83a5-a76673870246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a815f490d04c19af4646d7ed0f6e1f, user_id=8a8e1dd7b9354eb6b1ec882efd443dd6, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:00.777690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4569a2-68db-48f7-83a5-a76673870246 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:00.854727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1114/2269] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:00 2022] PUT /identity/v3/projects/69a815f490d04c19af4646d7ed0f6e1f/users/8a8e1dd7b9354eb6b1ec882efd443dd6/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:34:00.865878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3410d-8f5e-4130-b4b7-086f9428f741 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:00.866173 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3410d-8f5e-4130-b4b7-086f9428f741 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:00.866668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3410d-8f5e-4130-b4b7-086f9428f741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:00.907918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:00.911257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3fd3410d-8f5e-4130-b4b7-086f9428f741 None None] MFA Rules not processed for user `8a8e1dd7b9354eb6b1ec882efd443dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:00.915545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3fd3410d-8f5e-4130-b4b7-086f9428f741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:01.006059 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1156/2270] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:01.009109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4569a2-68db-48f7-83a5-a76673870246 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:01.084106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4569a2-68db-48f7-83a5-a76673870246 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:01.089794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4ba9f-a313-468c-903a-231591eaf62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:01.090461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4ba9f-a313-468c-903a-231591eaf62f admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:01.090830 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4ba9f-a313-468c-903a-231591eaf62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:01.091105 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4ba9f-a313-468c-903a-231591eaf62f admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:01.092189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4ba9f-a313-468c-903a-231591eaf62f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:01.093987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4ba9f-a313-468c-903a-231591eaf62f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:01.181378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1115/2271] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:34:01 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:01.185425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3410d-8f5e-4130-b4b7-086f9428f741 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:01.197400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3410d-8f5e-4130-b4b7-086f9428f741 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:01.285496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba08371-9440-4637-82b7-3ac996d9cec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:01.286161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba08371-9440-4637-82b7-3ac996d9cec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:01.286437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba08371-9440-4637-82b7-3ac996d9cec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:01.286745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba08371-9440-4637-82b7-3ac996d9cec8 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:01.287429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba08371-9440-4637-82b7-3ac996d9cec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:01.289543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba08371-9440-4637-82b7-3ac996d9cec8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:01.399079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1157/2272] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:34:01 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:01.403268 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4ba9f-a313-468c-903a-231591eaf62f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:01.414161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4ba9f-a313-468c-903a-231591eaf62f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:01.420324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa0449-8978-4319-9600-8f034bc11fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:01.420324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa0449-8978-4319-9600-8f034bc11fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:01.420324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa0449-8978-4319-9600-8f034bc11fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:01.420324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa0449-8978-4319-9600-8f034bc11fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:01.420856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa0449-8978-4319-9600-8f034bc11fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:01.422740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa0449-8978-4319-9600-8f034bc11fb3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:01.434166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1116/2273] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:01.438831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba08371-9440-4637-82b7-3ac996d9cec8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:01.449794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba08371-9440-4637-82b7-3ac996d9cec8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:01.455970 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5a10e-2a67-47e3-8dbf-e598c3b51b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:01.456607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5a10e-2a67-47e3-8dbf-e598c3b51b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:01.456875 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5a10e-2a67-47e3-8dbf-e598c3b51b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:01.457141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5a10e-2a67-47e3-8dbf-e598c3b51b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:01.457816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5a10e-2a67-47e3-8dbf-e598c3b51b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:01.459491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5a10e-2a67-47e3-8dbf-e598c3b51b38 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:01.471423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1158/2274] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:01.475881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa0449-8978-4319-9600-8f034bc11fb3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:01.487996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa0449-8978-4319-9600-8f034bc11fb3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:01.494623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e87e1-163d-4073-bb45-4667fcee0991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:01.495928 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e87e1-163d-4073-bb45-4667fcee0991 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:01.496207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e87e1-163d-4073-bb45-4667fcee0991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:01.496482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e87e1-163d-4073-bb45-4667fcee0991 admin admin] PATH_INFO: `/v3/projects/db9c8d3b982042acb7a403f3ca453ed3/users/23e175fe151e49e9b074167d31a1d38b/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:01.519865 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e87e1-163d-4073-bb45-4667fcee0991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9c8d3b982042acb7a403f3ca453ed3, user_id=23e175fe151e49e9b074167d31a1d38b, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:01.523123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e87e1-163d-4073-bb45-4667fcee0991 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:01.590037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1117/2275] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:01 2022] PUT /identity/v3/projects/db9c8d3b982042acb7a403f3ca453ed3/users/23e175fe151e49e9b074167d31a1d38b/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:01.604032 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd248-e79b-49e5-8c56-0644bd88e608 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:01.604404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd248-e79b-49e5-8c56-0644bd88e608 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:01.604929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd248-e79b-49e5-8c56-0644bd88e608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:01.634670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a13dd248-e79b-49e5-8c56-0644bd88e608 None None] MFA Rules not processed for user `23e175fe151e49e9b074167d31a1d38b`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:01.635914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a13dd248-e79b-49e5-8c56-0644bd88e608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:01.701154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1159/2276] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:01.705890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e87e1-163d-4073-bb45-4667fcee0991 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:01.769006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e87e1-163d-4073-bb45-4667fcee0991 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:01.773732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11328260-1305-4907-a926-3bed3a823734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:01.774326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11328260-1305-4907-a926-3bed3a823734 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:01.774600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11328260-1305-4907-a926-3bed3a823734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:01.774869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11328260-1305-4907-a926-3bed3a823734 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:01.775497 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11328260-1305-4907-a926-3bed3a823734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:01.777405 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11328260-1305-4907-a926-3bed3a823734 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:01.879404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1118/2277] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:34:01 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:01.883441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dd248-e79b-49e5-8c56-0644bd88e608 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:01.894820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dd248-e79b-49e5-8c56-0644bd88e608 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:01.952210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95adf9-a37f-4c78-8e64-d4e54e695b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:01.952210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95adf9-a37f-4c78-8e64-d4e54e695b55 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:01.952210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95adf9-a37f-4c78-8e64-d4e54e695b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:01.952210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95adf9-a37f-4c78-8e64-d4e54e695b55 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:01.952598 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95adf9-a37f-4c78-8e64-d4e54e695b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:01.954046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95adf9-a37f-4c78-8e64-d4e54e695b55 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:02.041396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1160/2278] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:34:01 2022] POST /identity/v3/users => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:02.045953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11328260-1305-4907-a926-3bed3a823734 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:02.051605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:02.057304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11328260-1305-4907-a926-3bed3a823734 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:02.062227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596a1fa-e667-497b-a974-0bdf7f078ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:02.062986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596a1fa-e667-497b-a974-0bdf7f078ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:02.063136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596a1fa-e667-497b-a974-0bdf7f078ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:02.063347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596a1fa-e667-497b-a974-0bdf7f078ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:02.063984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596a1fa-e667-497b-a974-0bdf7f078ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:02.065641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596a1fa-e667-497b-a974-0bdf7f078ad7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:02.078544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1119/2279] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:34:02.085970 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95adf9-a37f-4c78-8e64-d4e54e695b55 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:02.093232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95adf9-a37f-4c78-8e64-d4e54e695b55 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:02.097928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b8871-f7c6-492b-bbca-9a24feee7e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:02.098582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b8871-f7c6-492b-bbca-9a24feee7e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:02.098861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b8871-f7c6-492b-bbca-9a24feee7e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:02.099135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b8871-f7c6-492b-bbca-9a24feee7e61 admin admin] PATH_INFO: `/v3/projects/58a244a0cd254c6486127ed3b78998a7/users/e9395c2122db4ffca33c276a62da82f9/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:02.114088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b8871-f7c6-492b-bbca-9a24feee7e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a244a0cd254c6486127ed3b78998a7, user_id=e9395c2122db4ffca33c276a62da82f9, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:02.116933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b8871-f7c6-492b-bbca-9a24feee7e61 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:02.192496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1161/2280] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:02 2022] PUT /identity/v3/projects/58a244a0cd254c6486127ed3b78998a7/users/e9395c2122db4ffca33c276a62da82f9/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:02.196384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596a1fa-e667-497b-a974-0bdf7f078ad7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:02.206810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596a1fa-e667-497b-a974-0bdf7f078ad7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:02.262628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abbf89-18b0-4a34-87e7-586331420b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:02.263215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abbf89-18b0-4a34-87e7-586331420b31 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:02.263490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abbf89-18b0-4a34-87e7-586331420b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:02.263756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abbf89-18b0-4a34-87e7-586331420b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:02.265381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abbf89-18b0-4a34-87e7-586331420b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:02.267099 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abbf89-18b0-4a34-87e7-586331420b31 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:02.278385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1120/2281] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:02.281478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b8871-f7c6-492b-bbca-9a24feee7e61 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:02.291532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b8871-f7c6-492b-bbca-9a24feee7e61 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:02.295344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:02.298301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9294b-7148-4610-96bd-2daff4548165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:02.299097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9294b-7148-4610-96bd-2daff4548165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:02.299463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9294b-7148-4610-96bd-2daff4548165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:02.299761 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9294b-7148-4610-96bd-2daff4548165 admin admin] PATH_INFO: `/v3/projects/58a244a0cd254c6486127ed3b78998a7/users/e9395c2122db4ffca33c276a62da82f9/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:02.301736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9294b-7148-4610-96bd-2daff4548165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a244a0cd254c6486127ed3b78998a7, user_id=e9395c2122db4ffca33c276a62da82f9, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:02.304940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9294b-7148-4610-96bd-2daff4548165 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:02.316348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1162/2282] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:02 2022] PUT /identity/v3/projects/58a244a0cd254c6486127ed3b78998a7/users/e9395c2122db4ffca33c276a62da82f9/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:34:02.327432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089adfdc-4fee-42ba-9718-d5e5e966a892 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:02.327732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089adfdc-4fee-42ba-9718-d5e5e966a892 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:02.327979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089adfdc-4fee-42ba-9718-d5e5e966a892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:02.356616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-089adfdc-4fee-42ba-9718-d5e5e966a892 None None] MFA Rules not processed for user `e9395c2122db4ffca33c276a62da82f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:02.358567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-089adfdc-4fee-42ba-9718-d5e5e966a892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:02.431285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1121/2283] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:02.443809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9294b-7148-4610-96bd-2daff4548165 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:02.448228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5de9294b-7148-4610-96bd-2daff4548165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:02.506576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9294b-7148-4610-96bd-2daff4548165 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:02.510455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e3715-b541-4168-95b4-7d3ebb7ea4a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:02.511133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e3715-b541-4168-95b4-7d3ebb7ea4a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:02.511415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e3715-b541-4168-95b4-7d3ebb7ea4a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:02.511691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e3715-b541-4168-95b4-7d3ebb7ea4a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:02.579645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e3715-b541-4168-95b4-7d3ebb7ea4a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:02.581144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e3715-b541-4168-95b4-7d3ebb7ea4a9 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:02.591343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1163/2284] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:05.569779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089adfdc-4fee-42ba-9718-d5e5e966a892 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:05.572278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-089adfdc-4fee-42ba-9718-d5e5e966a892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:05.655240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089adfdc-4fee-42ba-9718-d5e5e966a892 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:05.659808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:05.660407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:05.660688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:05.660963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] PATH_INFO: `/v3/users/4872ec31feef423da52072f109969725` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:05.662532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4872ec31feef423da52072f109969725)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:05.664558 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:05.683629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:05.734845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e3715-b541-4168-95b4-7d3ebb7ea4a9 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:05.736695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-827e3715-b541-4168-95b4-7d3ebb7ea4a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:05.749966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e3715-b541-4168-95b4-7d3ebb7ea4a9 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:05.754276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a97b8-a38b-48ed-bf9e-e9ce5ca79cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:05.754886 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a97b8-a38b-48ed-bf9e-e9ce5ca79cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:05.755165 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a97b8-a38b-48ed-bf9e-e9ce5ca79cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:05.755437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a97b8-a38b-48ed-bf9e-e9ce5ca79cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:05.769156 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:05.814942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a97b8-a38b-48ed-bf9e-e9ce5ca79cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:05.816388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a97b8-a38b-48ed-bf9e-e9ce5ca79cc5 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:05.830067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1164/2285] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:34:05.888767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4872ec31feef423da52072f109969725'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:05.889431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4872ec31feef423da52072f109969725'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:05.963041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4872ec31feef423da52072f109969725'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:05.978377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4872ec31feef423da52072f109969725'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:05.988233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4872ec31feef423da52072f109969725'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:05.998011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1122/2286] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:05 2022] DELETE /identity/v3/users/4872ec31feef423da52072f109969725 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:05.999558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff95b6d-8811-4dcc-ad49-eac964c0bc07 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:05.999846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff95b6d-8811-4dcc-ad49-eac964c0bc07 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:06.000120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff95b6d-8811-4dcc-ad49-eac964c0bc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:06.020198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2ff95b6d-8811-4dcc-ad49-eac964c0bc07 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:06.021525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2ff95b6d-8811-4dcc-ad49-eac964c0bc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:06.087117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1165/2287] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:06.097740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:06.099387 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:06.176051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e544c-fde1-45e7-b356-8bc1c94896c4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:06.181368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e12b1c-2ec7-4064-aec4-7d63c92b06ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:06.182084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e12b1c-2ec7-4064-aec4-7d63c92b06ba service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:06.182474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e12b1c-2ec7-4064-aec4-7d63c92b06ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:06.182856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e12b1c-2ec7-4064-aec4-7d63c92b06ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:06.194064 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e12b1c-2ec7-4064-aec4-7d63c92b06ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:06.197796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e12b1c-2ec7-4064-aec4-7d63c92b06ba service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:06.210029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1123/2288] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:06.949868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff95b6d-8811-4dcc-ad49-eac964c0bc07 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:06.952142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2ff95b6d-8811-4dcc-ad49-eac964c0bc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:06.969537 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff95b6d-8811-4dcc-ad49-eac964c0bc07 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:06.980241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:06.980944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:06.981230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:06.981532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] PATH_INFO: `/v3/projects/b27dc488ebc24f9eb28aef05503a34e8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:06.983265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27dc488ebc24f9eb28aef05503a34e8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:06.986309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:07.190885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27dc488ebc24f9eb28aef05503a34e8', 'request_id': 'req-f3863941-d2f6-42e5-a1ee-79a307e6ead3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:07.271504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27dc488ebc24f9eb28aef05503a34e8', 'request_id': 'req-f3863941-d2f6-42e5-a1ee-79a307e6ead3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:07.278933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27dc488ebc24f9eb28aef05503a34e8', 'request_id': 'req-f3863941-d2f6-42e5-a1ee-79a307e6ead3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:07.292288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] The token cache is being invalidate because project b27dc488ebc24f9eb28aef05503a34e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:07.292573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:07.293227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:07.294772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1166/2289] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:06 2022] DELETE /identity/v3/projects/b27dc488ebc24f9eb28aef05503a34e8 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:07.299252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e12b1c-2ec7-4064-aec4-7d63c92b06ba service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:07.310108 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e12b1c-2ec7-4064-aec4-7d63c92b06ba service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:07.398147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:07.398879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:07.399182 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:07.399454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] PATH_INFO: `/v3/users/ac3c100f2c54467d8ccf374358a3da9e` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:07.400866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac3c100f2c54467d8ccf374358a3da9e)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:07.402864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:07.573046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:07.579358 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:07.609550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:07.623279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac3c100f2c54467d8ccf374358a3da9e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:07.623997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac3c100f2c54467d8ccf374358a3da9e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:07.634506 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:07.648399 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3863941-d2f6-42e5-a1ee-79a307e6ead3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:07.651984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e9619-a65e-46d8-ba7e-013677b83607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:07.652629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e9619-a65e-46d8-ba7e-013677b83607 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:07.652907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e9619-a65e-46d8-ba7e-013677b83607 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:07.653179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e9619-a65e-46d8-ba7e-013677b83607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:07.655529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e45e9619-a65e-46d8-ba7e-013677b83607 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:07.682377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac3c100f2c54467d8ccf374358a3da9e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:07.695098 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac3c100f2c54467d8ccf374358a3da9e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:07.702725 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac3c100f2c54467d8ccf374358a3da9e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:07.708749 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1124/2290] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:07 2022] DELETE /identity/v3/users/ac3c100f2c54467d8ccf374358a3da9e => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:07.724297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e9619-a65e-46d8-ba7e-013677b83607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:07.725889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e9619-a65e-46d8-ba7e-013677b83607 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:07.737152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1167/2291] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:34:08.122534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:08.124567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:08.197836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac316417-0be1-442e-be0f-8b484236d385 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:08.203273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:08.204038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:08.204317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:08.204587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] PATH_INFO: `/v3/projects/80c88965ebe8430da2f2b4221a618df9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:08.205799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] RBAC: Authorizing `identity:delete_project(project_id=80c88965ebe8430da2f2b4221a618df9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:08.207787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:08.324791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80c88965ebe8430da2f2b4221a618df9', 'request_id': 'req-506308d2-6702-48e5-9475-a1b68dbc905c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:08.391255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80c88965ebe8430da2f2b4221a618df9', 'request_id': 'req-506308d2-6702-48e5-9475-a1b68dbc905c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:08.398251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80c88965ebe8430da2f2b4221a618df9', 'request_id': 'req-506308d2-6702-48e5-9475-a1b68dbc905c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:08.409433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] The token cache is being invalidate because project 80c88965ebe8430da2f2b4221a618df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:08.409754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:08.410391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-506308d2-6702-48e5-9475-a1b68dbc905c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:08.411873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1125/2292] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:08 2022] DELETE /identity/v3/projects/80c88965ebe8430da2f2b4221a618df9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:08.417154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c7e67-7e12-4855-b4ff-5862c94bce16 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:08.417701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c7e67-7e12-4855-b4ff-5862c94bce16 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:08.417795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c7e67-7e12-4855-b4ff-5862c94bce16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:08.442080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e90c7e67-7e12-4855-b4ff-5862c94bce16 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:08.443310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e90c7e67-7e12-4855-b4ff-5862c94bce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:08.508195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1168/2293] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:08.520341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805c3ec-889a-4916-b700-f470d7ccea59 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:08.520623 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805c3ec-889a-4916-b700-f470d7ccea59 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:08.520892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805c3ec-889a-4916-b700-f470d7ccea59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:08.544254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5805c3ec-889a-4916-b700-f470d7ccea59 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:08.545581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5805c3ec-889a-4916-b700-f470d7ccea59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:08.553936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1126/2294] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:08.563741 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c7e67-7e12-4855-b4ff-5862c94bce16 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:08.573523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c7e67-7e12-4855-b4ff-5862c94bce16 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:08.579639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf823-fd92-4b03-ad32-7b85ebaab504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:08.580494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf823-fd92-4b03-ad32-7b85ebaab504 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:08.580780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf823-fd92-4b03-ad32-7b85ebaab504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:08.581049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf823-fd92-4b03-ad32-7b85ebaab504 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:08.581797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbf823-fd92-4b03-ad32-7b85ebaab504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:34:08.582099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbf823-fd92-4b03-ad32-7b85ebaab504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:08.583717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbf823-fd92-4b03-ad32-7b85ebaab504 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:08.596750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1169/2295] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:34:08.601421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805c3ec-889a-4916-b700-f470d7ccea59 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:08.611418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805c3ec-889a-4916-b700-f470d7ccea59 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:08.616417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3c0ea-ae17-4174-8c78-f34a1b637d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:08.617008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3c0ea-ae17-4174-8c78-f34a1b637d96 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:08.617286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3c0ea-ae17-4174-8c78-f34a1b637d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:08.617594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3c0ea-ae17-4174-8c78-f34a1b637d96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:08.618230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3c0ea-ae17-4174-8c78-f34a1b637d96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:08.620112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3c0ea-ae17-4174-8c78-f34a1b637d96 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:08.730725 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1127/2296] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:34:08 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:08.732840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf823-fd92-4b03-ad32-7b85ebaab504 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:08.745933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf823-fd92-4b03-ad32-7b85ebaab504 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:08.804827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62155a5-6fef-4ab0-bb21-cfe0aaa1e2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:08.805434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62155a5-6fef-4ab0-bb21-cfe0aaa1e2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:08.805743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62155a5-6fef-4ab0-bb21-cfe0aaa1e2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:08.806011 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62155a5-6fef-4ab0-bb21-cfe0aaa1e2c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:08.806644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62155a5-6fef-4ab0-bb21-cfe0aaa1e2c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:08.808555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62155a5-6fef-4ab0-bb21-cfe0aaa1e2c5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:08.917314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1170/2297] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:34:08 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:08.921915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3c0ea-ae17-4174-8c78-f34a1b637d96 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:08.932632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3c0ea-ae17-4174-8c78-f34a1b637d96 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:08.938009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6d5c8-8fc9-48e5-af01-0b2876161aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:08.938599 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6d5c8-8fc9-48e5-af01-0b2876161aeb admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:08.938872 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6d5c8-8fc9-48e5-af01-0b2876161aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:08.939138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6d5c8-8fc9-48e5-af01-0b2876161aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:08.939829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6d5c8-8fc9-48e5-af01-0b2876161aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:08.941439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6d5c8-8fc9-48e5-af01-0b2876161aeb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:08.952293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1128/2298] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:34:08.956646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62155a5-6fef-4ab0-bb21-cfe0aaa1e2c5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:08.966839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62155a5-6fef-4ab0-bb21-cfe0aaa1e2c5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:08.971790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bba47-a144-4e89-b75c-5c4a97f01485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:08.972398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bba47-a144-4e89-b75c-5c4a97f01485 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:08.972677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bba47-a144-4e89-b75c-5c4a97f01485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:08.972947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bba47-a144-4e89-b75c-5c4a97f01485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:08.973639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bba47-a144-4e89-b75c-5c4a97f01485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:08.975253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bba47-a144-4e89-b75c-5c4a97f01485 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:08.977053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:08.989142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1171/2299] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:08.992185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6d5c8-8fc9-48e5-af01-0b2876161aeb admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:08.999271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:09.002938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6d5c8-8fc9-48e5-af01-0b2876161aeb admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:09.007510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69bc2b-134b-4c71-9a03-63441a8d0b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:09.008162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69bc2b-134b-4c71-9a03-63441a8d0b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:09.008440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69bc2b-134b-4c71-9a03-63441a8d0b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:09.008709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69bc2b-134b-4c71-9a03-63441a8d0b6e admin admin] PATH_INFO: `/v3/projects/a080764e7f3b4746bcf95b71b16467e1/users/f035367170b6492d8451259f8e5d528f/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:09.024622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69bc2b-134b-4c71-9a03-63441a8d0b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a080764e7f3b4746bcf95b71b16467e1, user_id=f035367170b6492d8451259f8e5d528f, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:09.027316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69bc2b-134b-4c71-9a03-63441a8d0b6e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:09.109079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1129/2300] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:08 2022] PUT /identity/v3/projects/a080764e7f3b4746bcf95b71b16467e1/users/f035367170b6492d8451259f8e5d528f/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:09.111564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba248-ed6a-444d-af46-49d38ef6f8de None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:09.111852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba248-ed6a-444d-af46-49d38ef6f8de None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:09.112131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba248-ed6a-444d-af46-49d38ef6f8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:09.143479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8cdba248-ed6a-444d-af46-49d38ef6f8de None None] MFA Rules not processed for user `f035367170b6492d8451259f8e5d528f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:09.144717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8cdba248-ed6a-444d-af46-49d38ef6f8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:09.213640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1172/2301] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:09.219004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69bc2b-134b-4c71-9a03-63441a8d0b6e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:09.284144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69bc2b-134b-4c71-9a03-63441a8d0b6e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:09.288778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453569a5-2c94-44c0-a7f4-c8ad5020bbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:09.289365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453569a5-2c94-44c0-a7f4-c8ad5020bbd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:09.289676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453569a5-2c94-44c0-a7f4-c8ad5020bbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:09.289943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453569a5-2c94-44c0-a7f4-c8ad5020bbd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:09.290565 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453569a5-2c94-44c0-a7f4-c8ad5020bbd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:09.292452 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453569a5-2c94-44c0-a7f4-c8ad5020bbd7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:09.379844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1130/2302] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:34:09 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:34:09.383877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdba248-ed6a-444d-af46-49d38ef6f8de None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:09.395975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdba248-ed6a-444d-af46-49d38ef6f8de None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:09.460606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508246-50f0-4f75-8a36-6ed0d7c556c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:09.461276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508246-50f0-4f75-8a36-6ed0d7c556c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:09.461529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508246-50f0-4f75-8a36-6ed0d7c556c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:09.461856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508246-50f0-4f75-8a36-6ed0d7c556c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:09.462530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e508246-50f0-4f75-8a36-6ed0d7c556c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:09.464425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e508246-50f0-4f75-8a36-6ed0d7c556c9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:09.568555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1173/2303] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:34:09 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:09.572518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453569a5-2c94-44c0-a7f4-c8ad5020bbd7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:09.584649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453569a5-2c94-44c0-a7f4-c8ad5020bbd7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:09.595509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcf6d7-9693-4355-a944-064e80e5a245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:09.596204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcf6d7-9693-4355-a944-064e80e5a245 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:09.596484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcf6d7-9693-4355-a944-064e80e5a245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:09.596775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcf6d7-9693-4355-a944-064e80e5a245 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:09.597423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcf6d7-9693-4355-a944-064e80e5a245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:09.599079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcf6d7-9693-4355-a944-064e80e5a245 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:09.611305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1131/2304] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:09.616554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508246-50f0-4f75-8a36-6ed0d7c556c9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:09.628649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508246-50f0-4f75-8a36-6ed0d7c556c9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:09.633608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2dd4b-28d1-4bfd-9bef-5a2d297ed9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:09.634268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2dd4b-28d1-4bfd-9bef-5a2d297ed9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:09.634547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2dd4b-28d1-4bfd-9bef-5a2d297ed9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:09.634946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2dd4b-28d1-4bfd-9bef-5a2d297ed9d2 admin admin] PATH_INFO: `/v3/projects/f48240ed19764595b05c68455dc842a9/users/f53eb28630734b9a800f634ac2d28afd/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:09.652031 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2dd4b-28d1-4bfd-9bef-5a2d297ed9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48240ed19764595b05c68455dc842a9, user_id=f53eb28630734b9a800f634ac2d28afd, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:09.654787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2dd4b-28d1-4bfd-9bef-5a2d297ed9d2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:09.898280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1174/2305] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:09 2022] PUT /identity/v3/projects/f48240ed19764595b05c68455dc842a9/users/f53eb28630734b9a800f634ac2d28afd/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:09.902283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcf6d7-9693-4355-a944-064e80e5a245 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:09.913500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcf6d7-9693-4355-a944-064e80e5a245 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:09.973287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3af5c-a763-4eb6-a5f0-bcdec223a8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:09.973931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3af5c-a763-4eb6-a5f0-bcdec223a8df admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:09.974203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3af5c-a763-4eb6-a5f0-bcdec223a8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:09.974471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3af5c-a763-4eb6-a5f0-bcdec223a8df admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:09.975109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3af5c-a763-4eb6-a5f0-bcdec223a8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:09.976704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3af5c-a763-4eb6-a5f0-bcdec223a8df admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:09.987832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1132/2306] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:34:09.992232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2dd4b-28d1-4bfd-9bef-5a2d297ed9d2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:10.003481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2dd4b-28d1-4bfd-9bef-5a2d297ed9d2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:10.008674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4a4f5-1129-4762-a272-73f041acd5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:10.009630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4a4f5-1129-4762-a272-73f041acd5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:10.010012 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4a4f5-1129-4762-a272-73f041acd5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:10.011026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4a4f5-1129-4762-a272-73f041acd5ea admin admin] PATH_INFO: `/v3/projects/f48240ed19764595b05c68455dc842a9/users/f53eb28630734b9a800f634ac2d28afd/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:10.014240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4a4f5-1129-4762-a272-73f041acd5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48240ed19764595b05c68455dc842a9, user_id=f53eb28630734b9a800f634ac2d28afd, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:10.017705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4a4f5-1129-4762-a272-73f041acd5ea admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:10.028732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1175/2307] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:09 2022] PUT /identity/v3/projects/f48240ed19764595b05c68455dc842a9/users/f53eb28630734b9a800f634ac2d28afd/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:10.040135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec6fc9-c8b5-4952-8292-7f3c918bd849 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:10.040425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec6fc9-c8b5-4952-8292-7f3c918bd849 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:10.040704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec6fc9-c8b5-4952-8292-7f3c918bd849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:10.070144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7aec6fc9-c8b5-4952-8292-7f3c918bd849 None None] MFA Rules not processed for user `f53eb28630734b9a800f634ac2d28afd`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:10.071343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7aec6fc9-c8b5-4952-8292-7f3c918bd849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:10.112299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:10.141011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1133/2308] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:10.151731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4a4f5-1129-4762-a272-73f041acd5ea admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:10.153412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c2a4a4f5-1129-4762-a272-73f041acd5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:10.196411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:10.212564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4a4f5-1129-4762-a272-73f041acd5ea admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:10.216238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd6dfd-85c8-4e27-ab0b-616c75eb849f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:10.216840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd6dfd-85c8-4e27-ab0b-616c75eb849f service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:10.217127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd6dfd-85c8-4e27-ab0b-616c75eb849f service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:10.217398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd6dfd-85c8-4e27-ab0b-616c75eb849f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:10.227695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd6dfd-85c8-4e27-ab0b-616c75eb849f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:10.229132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd6dfd-85c8-4e27-ab0b-616c75eb849f service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:10.238697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1176/2309] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:11.355335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec6fc9-c8b5-4952-8292-7f3c918bd849 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:11.420228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec6fc9-c8b5-4952-8292-7f3c918bd849 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:11.425526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:11.426158 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:11.426437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:11.426710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] PATH_INFO: `/v3/users/f035367170b6492d8451259f8e5d528f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:11.427841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f035367170b6492d8451259f8e5d528f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:11.430075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:11.624408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f035367170b6492d8451259f8e5d528f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:11.624408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f035367170b6492d8451259f8e5d528f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:11.699672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f035367170b6492d8451259f8e5d528f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:11.715551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f035367170b6492d8451259f8e5d528f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:11.725140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f035367170b6492d8451259f8e5d528f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:11.731162 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1134/2310] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:11 2022] DELETE /identity/v3/users/f035367170b6492d8451259f8e5d528f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:11.735776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b914-f5e2-42b0-8e16-93bcceebc6ea None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:11.736207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b914-f5e2-42b0-8e16-93bcceebc6ea None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:11.736405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b914-f5e2-42b0-8e16-93bcceebc6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:11.761225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9503b914-f5e2-42b0-8e16-93bcceebc6ea None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:11.762687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9503b914-f5e2-42b0-8e16-93bcceebc6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:11.833174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1177/2311] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:11.843958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:11.846020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:11.910386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dca04f-fa0c-4d83-8293-7473c580665d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:11.914086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51791b-5840-4e6f-86c8-6ed155ae3277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:11.914872 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51791b-5840-4e6f-86c8-6ed155ae3277 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:11.915083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51791b-5840-4e6f-86c8-6ed155ae3277 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:11.915358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51791b-5840-4e6f-86c8-6ed155ae3277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:11.932022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51791b-5840-4e6f-86c8-6ed155ae3277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:11.932022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51791b-5840-4e6f-86c8-6ed155ae3277 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:11.932590 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:11.939463 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1135/2312] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:12.372317 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503b914-f5e2-42b0-8e16-93bcceebc6ea None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:12.374285 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9503b914-f5e2-42b0-8e16-93bcceebc6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:12.390637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503b914-f5e2-42b0-8e16-93bcceebc6ea None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:12.401273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:12.401273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:12.401273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:12.401273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] PATH_INFO: `/v3/projects/a080764e7f3b4746bcf95b71b16467e1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:12.403794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a080764e7f3b4746bcf95b71b16467e1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:12.404510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:12.522604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a080764e7f3b4746bcf95b71b16467e1', 'request_id': 'req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:12.583091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a080764e7f3b4746bcf95b71b16467e1', 'request_id': 'req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:12.591512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a080764e7f3b4746bcf95b71b16467e1', 'request_id': 'req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:12.602852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] The token cache is being invalidate because project a080764e7f3b4746bcf95b71b16467e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:12.603139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:12.603795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:12.605441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1178/2313] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:12 2022] DELETE /identity/v3/projects/a080764e7f3b4746bcf95b71b16467e1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:12.609400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51791b-5840-4e6f-86c8-6ed155ae3277 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:12.619944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51791b-5840-4e6f-86c8-6ed155ae3277 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:12.680955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:12.681693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:12.681902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:12.682176 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] PATH_INFO: `/v3/users/f53eb28630734b9a800f634ac2d28afd` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:12.683316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53eb28630734b9a800f634ac2d28afd)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:12.685277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:12.865975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53eb28630734b9a800f634ac2d28afd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:12.866780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53eb28630734b9a800f634ac2d28afd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:12.948527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53eb28630734b9a800f634ac2d28afd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:12.961787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53eb28630734b9a800f634ac2d28afd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:12.968808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53eb28630734b9a800f634ac2d28afd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:12.976396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1136/2314] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:12 2022] DELETE /identity/v3/users/f53eb28630734b9a800f634ac2d28afd => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:13.410232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:13.411686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:13.428949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87319f3d-ae51-4eee-8e1a-1ed6524fa2f8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:13.498591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:13.499256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:13.499535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:13.499806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] PATH_INFO: `/v3/projects/f48240ed19764595b05c68455dc842a9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:13.501666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48240ed19764595b05c68455dc842a9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:13.503651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:13.513785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:13.536481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:13.537996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:13.551228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c379e-8d90-4dc2-b68e-2e57784d5cca admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:13.556173 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:13.556787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:13.557075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:13.557339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] PATH_INFO: `/v3/users/8a8e1dd7b9354eb6b1ec882efd443dd6` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:13.561562 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8e1dd7b9354eb6b1ec882efd443dd6)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:13.564350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:13.616089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48240ed19764595b05c68455dc842a9', 'request_id': 'req-d089f443-1b35-43dc-8010-df85883737cb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:13.723200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48240ed19764595b05c68455dc842a9', 'request_id': 'req-d089f443-1b35-43dc-8010-df85883737cb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:13.734022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48240ed19764595b05c68455dc842a9', 'request_id': 'req-d089f443-1b35-43dc-8010-df85883737cb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:13.748812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] The token cache is being invalidate because project f48240ed19764595b05c68455dc842a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:13.748812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:13.748812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d089f443-1b35-43dc-8010-df85883737cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:13.748812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1179/2315] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:13 2022] DELETE /identity/v3/projects/f48240ed19764595b05c68455dc842a9 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:13.756735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7995379-1f38-43dd-a3c4-7665762484bc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:13.757015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7995379-1f38-43dd-a3c4-7665762484bc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:13.757294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7995379-1f38-43dd-a3c4-7665762484bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:13.778270 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e7995379-1f38-43dd-a3c4-7665762484bc None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:13.779469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e7995379-1f38-43dd-a3c4-7665762484bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:13.824853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8e1dd7b9354eb6b1ec882efd443dd6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:13.825950 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8e1dd7b9354eb6b1ec882efd443dd6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:13.841707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1180/2316] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:13.849860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cbc4-7830-4352-bcd0-7c368c1cfd5e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:13.850149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cbc4-7830-4352-bcd0-7c368c1cfd5e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:13.850425 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cbc4-7830-4352-bcd0-7c368c1cfd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:13.870091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0dd6cbc4-7830-4352-bcd0-7c368c1cfd5e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:13.871296 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0dd6cbc4-7830-4352-bcd0-7c368c1cfd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:13.881725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1181/2317] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:34:13.884297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6cbc4-7830-4352-bcd0-7c368c1cfd5e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:13.894260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6cbc4-7830-4352-bcd0-7c368c1cfd5e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:13.898960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564e604-3777-4671-88d3-28402b5af4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:13.899542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564e604-3777-4671-88d3-28402b5af4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:13.899820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564e604-3777-4671-88d3-28402b5af4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:13.900093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564e604-3777-4671-88d3-28402b5af4d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:13.900732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564e604-3777-4671-88d3-28402b5af4d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:34:13.901034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564e604-3777-4671-88d3-28402b5af4d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:13.902652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564e604-3777-4671-88d3-28402b5af4d2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:13.910595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8e1dd7b9354eb6b1ec882efd443dd6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:13.914961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1182/2318] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:34:13.920465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564e604-3777-4671-88d3-28402b5af4d2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:13.927993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8e1dd7b9354eb6b1ec882efd443dd6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:13.929399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:13.931520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564e604-3777-4671-88d3-28402b5af4d2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:13.935571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91d16121-4d59-4bb6-8b4c-206af1ee7838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8e1dd7b9354eb6b1ec882efd443dd6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:13.936497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255312c6-60c5-47f0-9103-8c3b9e9c0fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:13.937124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255312c6-60c5-47f0-9103-8c3b9e9c0fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:13.937627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255312c6-60c5-47f0-9103-8c3b9e9c0fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:13.937902 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255312c6-60c5-47f0-9103-8c3b9e9c0fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:13.938895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255312c6-60c5-47f0-9103-8c3b9e9c0fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:13.940793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255312c6-60c5-47f0-9103-8c3b9e9c0fe6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:13.941733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1137/2319] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:13 2022] DELETE /identity/v3/users/8a8e1dd7b9354eb6b1ec882efd443dd6 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:34:13.951391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f69f-35dd-4d42-9d36-ce38898fb8f8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:13.951665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f69f-35dd-4d42-9d36-ce38898fb8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:13.951939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f69f-35dd-4d42-9d36-ce38898fb8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:13.972940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-dc72f69f-35dd-4d42-9d36-ce38898fb8f8 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:13.977305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dc72f69f-35dd-4d42-9d36-ce38898fb8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:14.045914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1138/2320] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:34:14.055805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1183/2321] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:34:13 2022] POST /identity/v3/projects => generated 357 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:34:14.059811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72f69f-35dd-4d42-9d36-ce38898fb8f8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:14.061147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255312c6-60c5-47f0-9103-8c3b9e9c0fe6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:14.061340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dc72f69f-35dd-4d42-9d36-ce38898fb8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:14.115649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:14.120022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72f69f-35dd-4d42-9d36-ce38898fb8f8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:14.126728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255312c6-60c5-47f0-9103-8c3b9e9c0fe6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:14.131440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2df1c-44e1-4d15-a6b5-5ac4bafc9653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:14.132069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2df1c-44e1-4d15-a6b5-5ac4bafc9653 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:14.132320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2df1c-44e1-4d15-a6b5-5ac4bafc9653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:14.132604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2df1c-44e1-4d15-a6b5-5ac4bafc9653 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:14.133188 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2df1c-44e1-4d15-a6b5-5ac4bafc9653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:14.135136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2df1c-44e1-4d15-a6b5-5ac4bafc9653 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:14.172066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21186a1c-4a48-4ac2-ba16-a48fb5e24fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:14.172721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21186a1c-4a48-4ac2-ba16-a48fb5e24fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:14.173186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21186a1c-4a48-4ac2-ba16-a48fb5e24fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:14.173489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21186a1c-4a48-4ac2-ba16-a48fb5e24fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:14.183618 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21186a1c-4a48-4ac2-ba16-a48fb5e24fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:14.185062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21186a1c-4a48-4ac2-ba16-a48fb5e24fa6 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:14.196119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1139/2322] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:14.234360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1184/2323] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:34:14 2022] POST /identity/v3/users => generated 279 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:14.239232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21186a1c-4a48-4ac2-ba16-a48fb5e24fa6 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:14.249036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21186a1c-4a48-4ac2-ba16-a48fb5e24fa6 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:14.255636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f996e8-9f62-4940-a7d9-976d9d55cea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:14.256347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f996e8-9f62-4940-a7d9-976d9d55cea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:14.256703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f996e8-9f62-4940-a7d9-976d9d55cea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:14.257041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f996e8-9f62-4940-a7d9-976d9d55cea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:14.257727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f996e8-9f62-4940-a7d9-976d9d55cea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:14.259375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f996e8-9f62-4940-a7d9-976d9d55cea4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:14.270271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1140/2324] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:14.274401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2df1c-44e1-4d15-a6b5-5ac4bafc9653 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:14.285784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2df1c-44e1-4d15-a6b5-5ac4bafc9653 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:14.290503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2651f8a-2dff-4bea-8cfb-c81107451559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:14.291097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651f8a-2dff-4bea-8cfb-c81107451559 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:14.291376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651f8a-2dff-4bea-8cfb-c81107451559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:14.291645 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651f8a-2dff-4bea-8cfb-c81107451559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:14.292288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2651f8a-2dff-4bea-8cfb-c81107451559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:14.293908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2651f8a-2dff-4bea-8cfb-c81107451559 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:14.308015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1185/2325] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:14.310431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f996e8-9f62-4940-a7d9-976d9d55cea4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:14.321164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f996e8-9f62-4940-a7d9-976d9d55cea4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:14.325738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be1742-e349-41f6-a6f7-2958578d9f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:14.326391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be1742-e349-41f6-a6f7-2958578d9f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:14.326661 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be1742-e349-41f6-a6f7-2958578d9f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:14.326931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be1742-e349-41f6-a6f7-2958578d9f18 admin admin] PATH_INFO: `/v3/projects/d775ff50a7b94cccaae9e998c0af0889/users/02c5978cf5594bceb720e66baa1f3f80/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:14.341963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be1742-e349-41f6-a6f7-2958578d9f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d775ff50a7b94cccaae9e998c0af0889, user_id=02c5978cf5594bceb720e66baa1f3f80, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:14.345510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be1742-e349-41f6-a6f7-2958578d9f18 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:14.466585 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1141/2326] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:14 2022] PUT /identity/v3/projects/d775ff50a7b94cccaae9e998c0af0889/users/02c5978cf5594bceb720e66baa1f3f80/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:14.477721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1084053-5c96-43e4-9795-ffd71284a51c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:14.478420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1084053-5c96-43e4-9795-ffd71284a51c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:14.478420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1084053-5c96-43e4-9795-ffd71284a51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:14.507010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f1084053-5c96-43e4-9795-ffd71284a51c None None] MFA Rules not processed for user `02c5978cf5594bceb720e66baa1f3f80`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:14.508234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f1084053-5c96-43e4-9795-ffd71284a51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:14.580572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1186/2327] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:34:14.614788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be1742-e349-41f6-a6f7-2958578d9f18 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:14.617040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-65be1742-e349-41f6-a6f7-2958578d9f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:14.626861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1084053-5c96-43e4-9795-ffd71284a51c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:14.629826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f1084053-5c96-43e4-9795-ffd71284a51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:14.640607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:14.691563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be1742-e349-41f6-a6f7-2958578d9f18 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:14.695396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc077c-d098-4f0f-b50f-b4b538e5ca14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:14.696126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc077c-d098-4f0f-b50f-b4b538e5ca14 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:14.696751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc077c-d098-4f0f-b50f-b4b538e5ca14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:14.699857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc077c-d098-4f0f-b50f-b4b538e5ca14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:14.706916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1084053-5c96-43e4-9795-ffd71284a51c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:14.708339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc077c-d098-4f0f-b50f-b4b538e5ca14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:14.709884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc077c-d098-4f0f-b50f-b4b538e5ca14 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:14.711687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:14.712333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:14.712672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:14.712906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] PATH_INFO: `/v3/projects/69a815f490d04c19af4646d7ed0f6e1f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:14.715385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a815f490d04c19af4646d7ed0f6e1f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:14.716630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:14.721430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1142/2328] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:14.739844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc077c-d098-4f0f-b50f-b4b538e5ca14 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:14.753236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc077c-d098-4f0f-b50f-b4b538e5ca14 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:14.757947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:14.764202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:14.764202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:14.764202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] PATH_INFO: `/v3/users/02c5978cf5594bceb720e66baa1f3f80` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:14.764202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c5978cf5594bceb720e66baa1f3f80)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:14.764202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:14.831866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a815f490d04c19af4646d7ed0f6e1f', 'request_id': 'req-8d1628d7-7527-484c-92bf-30643a0c6e38'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:14.898996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a815f490d04c19af4646d7ed0f6e1f', 'request_id': 'req-8d1628d7-7527-484c-92bf-30643a0c6e38'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:14.908546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a815f490d04c19af4646d7ed0f6e1f', 'request_id': 'req-8d1628d7-7527-484c-92bf-30643a0c6e38'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:14.910939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:14.919426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] The token cache is being invalidate because project 69a815f490d04c19af4646d7ed0f6e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:14.919709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:14.920428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:14.921970 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1187/2329] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:14 2022] DELETE /identity/v3/projects/69a815f490d04c19af4646d7ed0f6e1f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:34:14.926438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:14.927849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:14.942316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c5978cf5594bceb720e66baa1f3f80'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:14.942980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c5978cf5594bceb720e66baa1f3f80'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:14.994640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1628d7-7527-484c-92bf-30643a0c6e38 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:15.001941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:15.002593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:15.002866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:15.003131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] PATH_INFO: `/v3/users/23e175fe151e49e9b074167d31a1d38b` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:15.004287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23e175fe151e49e9b074167d31a1d38b)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:15.006249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:15.013349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c5978cf5594bceb720e66baa1f3f80'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:15.028264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c5978cf5594bceb720e66baa1f3f80'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:15.036199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-df42612f-9baf-41a2-af52-f16a008c330b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c5978cf5594bceb720e66baa1f3f80'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:15.042441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1143/2330] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:14 2022] DELETE /identity/v3/users/02c5978cf5594bceb720e66baa1f3f80 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:15.046743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdccb7-18ae-4fe4-9f27-b195ea6130db None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:15.047122 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdccb7-18ae-4fe4-9f27-b195ea6130db None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:15.048311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdccb7-18ae-4fe4-9f27-b195ea6130db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:15.072174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8bfdccb7-18ae-4fe4-9f27-b195ea6130db None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:15.073404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8bfdccb7-18ae-4fe4-9f27-b195ea6130db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:15.133449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1144/2331] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:34:15.144542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdccb7-18ae-4fe4-9f27-b195ea6130db None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:15.145980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8bfdccb7-18ae-4fe4-9f27-b195ea6130db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:15.178885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:15.202871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23e175fe151e49e9b074167d31a1d38b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:15.207444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdccb7-18ae-4fe4-9f27-b195ea6130db None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:15.211306 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5016f0-01d9-45bf-b655-d237a542a9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:15.212271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5016f0-01d9-45bf-b655-d237a542a9bc service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:15.212720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5016f0-01d9-45bf-b655-d237a542a9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:15.213210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5016f0-01d9-45bf-b655-d237a542a9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:15.215956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23e175fe151e49e9b074167d31a1d38b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:15.224125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23e175fe151e49e9b074167d31a1d38b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:15.228053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5016f0-01d9-45bf-b655-d237a542a9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:15.233416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5016f0-01d9-45bf-b655-d237a542a9bc service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:15.239262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1145/2332] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:34:15.311046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23e175fe151e49e9b074167d31a1d38b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:15.311994 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23e175fe151e49e9b074167d31a1d38b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:15.318944 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1188/2333] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:14 2022] DELETE /identity/v3/users/23e175fe151e49e9b074167d31a1d38b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:34:15.701717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5016f0-01d9-45bf-b655-d237a542a9bc service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:15.703333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fe5016f0-01d9-45bf-b655-d237a542a9bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:15.722978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5016f0-01d9-45bf-b655-d237a542a9bc service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:15.746511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:15.748103 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:15.808484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:15.809121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:15.814629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:15.814629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] PATH_INFO: `/v3/projects/d775ff50a7b94cccaae9e998c0af0889` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:15.814629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d775ff50a7b94cccaae9e998c0af0889)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:15.814629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:15.822758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6841a-b7ca-4823-b414-a10080924e85 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:15.827510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:15.828139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:15.828418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:15.829272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] PATH_INFO: `/v3/projects/db9c8d3b982042acb7a403f3ca453ed3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:15.830507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] RBAC: Authorizing `identity:delete_project(project_id=db9c8d3b982042acb7a403f3ca453ed3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:15.832450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:15.849076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:15.968901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd775ff50a7b94cccaae9e998c0af0889', 'request_id': 'req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:15.979731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db9c8d3b982042acb7a403f3ca453ed3', 'request_id': 'req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.070737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd775ff50a7b94cccaae9e998c0af0889', 'request_id': 'req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.078010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd775ff50a7b94cccaae9e998c0af0889', 'request_id': 'req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.089420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] The token cache is being invalidate because project d775ff50a7b94cccaae9e998c0af0889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:16.089731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.090370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-26025e92-dd4c-4f1f-a2a0-0a2c04023e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.091893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1146/2334] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:15 2022] DELETE /identity/v3/projects/d775ff50a7b94cccaae9e998c0af0889 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:16.097116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb09472-aaf1-4850-b56f-3acc9bb0e71a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:16.097408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb09472-aaf1-4850-b56f-3acc9bb0e71a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:16.097701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb09472-aaf1-4850-b56f-3acc9bb0e71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:16.118018 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-dcb09472-aaf1-4850-b56f-3acc9bb0e71a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:16.119217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dcb09472-aaf1-4850-b56f-3acc9bb0e71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:16.155939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db9c8d3b982042acb7a403f3ca453ed3', 'request_id': 'req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.163388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db9c8d3b982042acb7a403f3ca453ed3', 'request_id': 'req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.174870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] The token cache is being invalidate because project db9c8d3b982042acb7a403f3ca453ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:16.175275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.175923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.177477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1189/2335] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:15 2022] DELETE /identity/v3/projects/db9c8d3b982042acb7a403f3ca453ed3 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:34:16.181193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1147/2336] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:16.182008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:16.184269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:16.190101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf49a8f-b951-4072-aca9-98db20571856 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:16.190383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf49a8f-b951-4072-aca9-98db20571856 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:16.190651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf49a8f-b951-4072-aca9-98db20571856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:16.198503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38eb92-e0ea-40ba-a57a-ed9dfbacd89f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:16.201288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:16.204254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:16.204928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:16.205211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:16.205528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] PATH_INFO: `/v3/users/e9395c2122db4ffca33c276a62da82f9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:16.206755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9395c2122db4ffca33c276a62da82f9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:16.208682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:16.211476 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ecf49a8f-b951-4072-aca9-98db20571856 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:16.212663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ecf49a8f-b951-4072-aca9-98db20571856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:16.221549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1148/2337] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:16.225706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf49a8f-b951-4072-aca9-98db20571856 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:16.235738 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf49a8f-b951-4072-aca9-98db20571856 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:16.240326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22e28f-1b16-427c-bd58-1d3b9fd1f585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:16.240930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22e28f-1b16-427c-bd58-1d3b9fd1f585 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:16.241206 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22e28f-1b16-427c-bd58-1d3b9fd1f585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:16.241500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22e28f-1b16-427c-bd58-1d3b9fd1f585 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:16.242181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22e28f-1b16-427c-bd58-1d3b9fd1f585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:34:16.242481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22e28f-1b16-427c-bd58-1d3b9fd1f585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:16.244238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22e28f-1b16-427c-bd58-1d3b9fd1f585 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:16.256461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1149/2338] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:34:16.261035 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22e28f-1b16-427c-bd58-1d3b9fd1f585 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:16.271168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22e28f-1b16-427c-bd58-1d3b9fd1f585 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:16.276416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b508ea-2adb-4624-a1b3-04175f0d3469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:16.277171 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b508ea-2adb-4624-a1b3-04175f0d3469 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:16.277468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b508ea-2adb-4624-a1b3-04175f0d3469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:16.277764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b508ea-2adb-4624-a1b3-04175f0d3469 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:16.278392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b508ea-2adb-4624-a1b3-04175f0d3469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:16.280358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b508ea-2adb-4624-a1b3-04175f0d3469 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:16.396709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1150/2339] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:34:16 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:34:16.401002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b508ea-2adb-4624-a1b3-04175f0d3469 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:16.426833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9395c2122db4ffca33c276a62da82f9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.441051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9395c2122db4ffca33c276a62da82f9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.449991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9395c2122db4ffca33c276a62da82f9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.466501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b508ea-2adb-4624-a1b3-04175f0d3469 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:16.471898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da003fd3-4682-4493-a160-a6bac023e0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:16.472438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da003fd3-4682-4493-a160-a6bac023e0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:16.472714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da003fd3-4682-4493-a160-a6bac023e0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:16.473006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da003fd3-4682-4493-a160-a6bac023e0c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:16.473679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da003fd3-4682-4493-a160-a6bac023e0c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:16.475587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da003fd3-4682-4493-a160-a6bac023e0c2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:16.508403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9395c2122db4ffca33c276a62da82f9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.509046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9395c2122db4ffca33c276a62da82f9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:16.515567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1190/2340] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:16 2022] DELETE /identity/v3/users/e9395c2122db4ffca33c276a62da82f9 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:16.572603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1151/2341] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:34:16 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:34:16.576875 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:16.578372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:16.648007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a567b-0254-47d6-a506-be44f81cef31 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:16.652576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099a94d-05cb-4406-a53c-5700e87e05e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:16.653179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a94d-05cb-4406-a53c-5700e87e05e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:16.653482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a94d-05cb-4406-a53c-5700e87e05e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:16.653792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a94d-05cb-4406-a53c-5700e87e05e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:16.654404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099a94d-05cb-4406-a53c-5700e87e05e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:16.656021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099a94d-05cb-4406-a53c-5700e87e05e4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:16.666320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1191/2342] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:34:16.671347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da003fd3-4682-4493-a160-a6bac023e0c2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:16.682144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da003fd3-4682-4493-a160-a6bac023e0c2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:16.686735 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f81cc0-6eb7-4b6b-9bc9-66007948c132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:16.687330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f81cc0-6eb7-4b6b-9bc9-66007948c132 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:16.687610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f81cc0-6eb7-4b6b-9bc9-66007948c132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:16.688036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f81cc0-6eb7-4b6b-9bc9-66007948c132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:16.688668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f81cc0-6eb7-4b6b-9bc9-66007948c132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:16.690292 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f81cc0-6eb7-4b6b-9bc9-66007948c132 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:16.700112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1152/2343] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:16.705223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099a94d-05cb-4406-a53c-5700e87e05e4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:16.720298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099a94d-05cb-4406-a53c-5700e87e05e4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:16.727905 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f44bd-3955-4998-becb-d740c7dd2c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:16.728586 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f44bd-3955-4998-becb-d740c7dd2c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:16.728813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f44bd-3955-4998-becb-d740c7dd2c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:16.729178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f44bd-3955-4998-becb-d740c7dd2c2c admin admin] PATH_INFO: `/v3/projects/515b093a56fa4ffaabbb3b136ec5c553/users/ba7eb809a33c4333ad076ba4c4cf0e50/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:16.735301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:16.751400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f44bd-3955-4998-becb-d740c7dd2c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=515b093a56fa4ffaabbb3b136ec5c553, user_id=ba7eb809a33c4333ad076ba4c4cf0e50, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:16.754648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f44bd-3955-4998-becb-d740c7dd2c2c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:16.843056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1192/2344] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:16 2022] PUT /identity/v3/projects/515b093a56fa4ffaabbb3b136ec5c553/users/ba7eb809a33c4333ad076ba4c4cf0e50/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:34:16.853770 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fde6d-ab5e-4251-b9d9-fc94c226ed7e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:16.854060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fde6d-ab5e-4251-b9d9-fc94c226ed7e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:16.854344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fde6d-ab5e-4251-b9d9-fc94c226ed7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:16.884582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a12fde6d-ab5e-4251-b9d9-fc94c226ed7e None None] MFA Rules not processed for user `ba7eb809a33c4333ad076ba4c4cf0e50`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:16.885843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a12fde6d-ab5e-4251-b9d9-fc94c226ed7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:16.958442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1153/2345] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:16.971157 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f44bd-3955-4998-becb-d740c7dd2c2c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:16.972932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d65f44bd-3955-4998-becb-d740c7dd2c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:16.984178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fde6d-ab5e-4251-b9d9-fc94c226ed7e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:16.984178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a12fde6d-ab5e-4251-b9d9-fc94c226ed7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:17.024333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:17.048996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f44bd-3955-4998-becb-d740c7dd2c2c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:17.052733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26568fc8-adef-442d-86bd-8d9b2e25a461 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:17.053338 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26568fc8-adef-442d-86bd-8d9b2e25a461 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:17.053721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26568fc8-adef-442d-86bd-8d9b2e25a461 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:17.054245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26568fc8-adef-442d-86bd-8d9b2e25a461 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:17.058980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fde6d-ab5e-4251-b9d9-fc94c226ed7e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:17.063944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:17.064094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26568fc8-adef-442d-86bd-8d9b2e25a461 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:17.064566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:17.064845 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:17.065119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] PATH_INFO: `/v3/projects/58a244a0cd254c6486127ed3b78998a7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:17.065519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26568fc8-adef-442d-86bd-8d9b2e25a461 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:17.066307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a244a0cd254c6486127ed3b78998a7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:17.068250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:17.074723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1193/2346] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:17.186237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a244a0cd254c6486127ed3b78998a7', 'request_id': 'req-3c27a751-ce1b-4597-92e6-edbe1f3c2585'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:17.245982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a244a0cd254c6486127ed3b78998a7', 'request_id': 'req-3c27a751-ce1b-4597-92e6-edbe1f3c2585'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:17.253562 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a244a0cd254c6486127ed3b78998a7', 'request_id': 'req-3c27a751-ce1b-4597-92e6-edbe1f3c2585'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:17.264193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] The token cache is being invalidate because project 58a244a0cd254c6486127ed3b78998a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:17.264482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:17.265158 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3c27a751-ce1b-4597-92e6-edbe1f3c2585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:17.267293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1154/2347] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:16 2022] DELETE /identity/v3/projects/58a244a0cd254c6486127ed3b78998a7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:17.272331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0489b2-ee03-4fb5-a991-d1bdf4794e15 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:17.272621 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0489b2-ee03-4fb5-a991-d1bdf4794e15 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:17.272899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0489b2-ee03-4fb5-a991-d1bdf4794e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:17.293936 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3e0489b2-ee03-4fb5-a991-d1bdf4794e15 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:17.295146 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3e0489b2-ee03-4fb5-a991-d1bdf4794e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:17.357054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1194/2348] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:17.365366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4624bc-b77e-4a03-b83b-cd52ab598b9c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:17.365692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4624bc-b77e-4a03-b83b-cd52ab598b9c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:17.365969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4624bc-b77e-4a03-b83b-cd52ab598b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:17.388218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5d4624bc-b77e-4a03-b83b-cd52ab598b9c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:17.390396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5d4624bc-b77e-4a03-b83b-cd52ab598b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:17.399980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1155/2349] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:17.404371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0489b2-ee03-4fb5-a991-d1bdf4794e15 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:17.415076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0489b2-ee03-4fb5-a991-d1bdf4794e15 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:17.419861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83505b7b-82e0-4179-9b37-d4f3e04a491c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:17.420456 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83505b7b-82e0-4179-9b37-d4f3e04a491c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:17.420734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83505b7b-82e0-4179-9b37-d4f3e04a491c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:17.421004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83505b7b-82e0-4179-9b37-d4f3e04a491c admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:17.421690 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83505b7b-82e0-4179-9b37-d4f3e04a491c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:34:17.421996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83505b7b-82e0-4179-9b37-d4f3e04a491c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:17.424300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83505b7b-82e0-4179-9b37-d4f3e04a491c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:17.440048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1195/2350] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:34:17.444448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4624bc-b77e-4a03-b83b-cd52ab598b9c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:17.453795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4624bc-b77e-4a03-b83b-cd52ab598b9c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:17.460050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0b720-ae6a-4da8-97db-c88c7b54bd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:17.460652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0b720-ae6a-4da8-97db-c88c7b54bd3a admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:17.460930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0b720-ae6a-4da8-97db-c88c7b54bd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:17.461199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0b720-ae6a-4da8-97db-c88c7b54bd3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:17.461862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0b720-ae6a-4da8-97db-c88c7b54bd3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:17.463863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0b720-ae6a-4da8-97db-c88c7b54bd3a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:17.547720 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1156/2351] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:34:17 2022] POST /identity/v3/projects => generated 399 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:17.551201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83505b7b-82e0-4179-9b37-d4f3e04a491c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:17.563569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83505b7b-82e0-4179-9b37-d4f3e04a491c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:17.626555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5733582-3e74-4761-a24d-076a29b9d81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:17.627485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5733582-3e74-4761-a24d-076a29b9d81d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:17.627975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5733582-3e74-4761-a24d-076a29b9d81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:17.628431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5733582-3e74-4761-a24d-076a29b9d81d admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:17.630301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5733582-3e74-4761-a24d-076a29b9d81d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:17.635175 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5733582-3e74-4761-a24d-076a29b9d81d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:17.715080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1196/2352] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:34:17 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:17.720318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0b720-ae6a-4da8-97db-c88c7b54bd3a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:17.731189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0b720-ae6a-4da8-97db-c88c7b54bd3a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:17.735967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b8e20-d680-4aae-924a-386fb7916c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:17.736571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b8e20-d680-4aae-924a-386fb7916c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:17.736844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b8e20-d680-4aae-924a-386fb7916c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:17.737110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b8e20-d680-4aae-924a-386fb7916c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:17.737795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b8e20-d680-4aae-924a-386fb7916c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:17.739400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b8e20-d680-4aae-924a-386fb7916c2d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:17.749750 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1157/2353] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:17.754122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5733582-3e74-4761-a24d-076a29b9d81d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:17.760479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:17.764716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5733582-3e74-4761-a24d-076a29b9d81d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:17.770484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827ee97-70d6-411b-82b1-cfdde71ecbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:17.771217 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827ee97-70d6-411b-82b1-cfdde71ecbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:17.771780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827ee97-70d6-411b-82b1-cfdde71ecbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:17.772128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827ee97-70d6-411b-82b1-cfdde71ecbe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:17.773486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827ee97-70d6-411b-82b1-cfdde71ecbe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:17.774629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827ee97-70d6-411b-82b1-cfdde71ecbe8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:17.787640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1197/2354] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:17.791914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b8e20-d680-4aae-924a-386fb7916c2d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:17.802216 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b8e20-d680-4aae-924a-386fb7916c2d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:17.807303 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7ab5e-4a7b-458d-9b90-a3446655c2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:17.807957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7ab5e-4a7b-458d-9b90-a3446655c2a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:17.808231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7ab5e-4a7b-458d-9b90-a3446655c2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:17.808510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7ab5e-4a7b-458d-9b90-a3446655c2a9 admin admin] PATH_INFO: `/v3/projects/fa2b6fc441fa4222a6b9ac498c4910c7/users/b0a80d355be84cb7946c5dd2737af2ab/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:17.829186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7ab5e-4a7b-458d-9b90-a3446655c2a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2b6fc441fa4222a6b9ac498c4910c7, user_id=b0a80d355be84cb7946c5dd2737af2ab, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:17.831938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7ab5e-4a7b-458d-9b90-a3446655c2a9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:17.934946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1158/2355] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:17 2022] PUT /identity/v3/projects/fa2b6fc441fa4222a6b9ac498c4910c7/users/b0a80d355be84cb7946c5dd2737af2ab/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:18.004682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbeec24-83fe-47b1-8221-9ccaa67b59c1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:18.004984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbeec24-83fe-47b1-8221-9ccaa67b59c1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:18.005259 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbeec24-83fe-47b1-8221-9ccaa67b59c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:18.033859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0bbeec24-83fe-47b1-8221-9ccaa67b59c1 None None] MFA Rules not processed for user `b0a80d355be84cb7946c5dd2737af2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:18.035060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0bbeec24-83fe-47b1-8221-9ccaa67b59c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:18.101840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1198/2356] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:18.113504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7ab5e-4a7b-458d-9b90-a3446655c2a9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:18.115311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e5c7ab5e-4a7b-458d-9b90-a3446655c2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:18.175353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7ab5e-4a7b-458d-9b90-a3446655c2a9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:18.179348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0bc1d-4fe3-4262-a50a-6d9ddf4b671e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:18.179949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0bc1d-4fe3-4262-a50a-6d9ddf4b671e service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:18.180225 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0bc1d-4fe3-4262-a50a-6d9ddf4b671e service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:18.180499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0bc1d-4fe3-4262-a50a-6d9ddf4b671e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:18.189308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0bc1d-4fe3-4262-a50a-6d9ddf4b671e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:18.190799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0bc1d-4fe3-4262-a50a-6d9ddf4b671e service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:18.194372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:18.200451 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1159/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:21.178335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbeec24-83fe-47b1-8221-9ccaa67b59c1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:21.180020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0bbeec24-83fe-47b1-8221-9ccaa67b59c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:21.247154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbeec24-83fe-47b1-8221-9ccaa67b59c1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:21.251706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:21.252327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:21.252607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:21.252873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] PATH_INFO: `/v3/users/ba7eb809a33c4333ad076ba4c4cf0e50` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:21.253998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba7eb809a33c4333ad076ba4c4cf0e50)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:21.256331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:21.490167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba7eb809a33c4333ad076ba4c4cf0e50'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:21.502339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba7eb809a33c4333ad076ba4c4cf0e50'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:21.510223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba7eb809a33c4333ad076ba4c4cf0e50'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:21.567281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba7eb809a33c4333ad076ba4c4cf0e50'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:21.569055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba7eb809a33c4333ad076ba4c4cf0e50'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:21.576279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1199/2358] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:21 2022] DELETE /identity/v3/users/ba7eb809a33c4333ad076ba4c4cf0e50 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:34:21.580789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76de17-18bf-4064-be67-f0fe454d8e28 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:21.581256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76de17-18bf-4064-be67-f0fe454d8e28 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:21.581350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76de17-18bf-4064-be67-f0fe454d8e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:21.602418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-dc76de17-18bf-4064-be67-f0fe454d8e28 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:21.603729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dc76de17-18bf-4064-be67-f0fe454d8e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:21.664589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1160/2359] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:21.674253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:21.675757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:21.681510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:21.736418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a6a96-d817-4be7-a292-711050531051 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:21.740261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8486556-471d-43ab-a295-180772f99441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:21.740865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8486556-471d-43ab-a295-180772f99441 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:21.741142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8486556-471d-43ab-a295-180772f99441 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:21.741409 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8486556-471d-43ab-a295-180772f99441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:21.750242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8486556-471d-43ab-a295-180772f99441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:21.751688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8486556-471d-43ab-a295-180772f99441 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:21.762283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1200/2360] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:22.163931 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76de17-18bf-4064-be67-f0fe454d8e28 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:22.165415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dc76de17-18bf-4064-be67-f0fe454d8e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:22.186581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76de17-18bf-4064-be67-f0fe454d8e28 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:22.192008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:22.193215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:22.193539 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:22.193858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] PATH_INFO: `/v3/projects/515b093a56fa4ffaabbb3b136ec5c553` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:22.195088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=515b093a56fa4ffaabbb3b136ec5c553)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:22.197180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:22.302664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '515b093a56fa4ffaabbb3b136ec5c553', 'request_id': 'req-d61a7fc2-c715-4f38-849f-4181188bfde0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:22.376324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '515b093a56fa4ffaabbb3b136ec5c553', 'request_id': 'req-d61a7fc2-c715-4f38-849f-4181188bfde0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:22.383416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '515b093a56fa4ffaabbb3b136ec5c553', 'request_id': 'req-d61a7fc2-c715-4f38-849f-4181188bfde0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:22.393756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] The token cache is being invalidate because project 515b093a56fa4ffaabbb3b136ec5c553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:22.394039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:22.394660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d61a7fc2-c715-4f38-849f-4181188bfde0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:22.396756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1161/2361] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:22 2022] DELETE /identity/v3/projects/515b093a56fa4ffaabbb3b136ec5c553 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:22.401567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eece09-1068-4304-afdf-ca44684cac70 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:22.402038 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eece09-1068-4304-afdf-ca44684cac70 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:22.402150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eece09-1068-4304-afdf-ca44684cac70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:22.422352 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b6eece09-1068-4304-afdf-ca44684cac70 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:22.423480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b6eece09-1068-4304-afdf-ca44684cac70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:22.481909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1201/2362] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:22.490038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba1807-bef8-459a-8dd3-c1ad704cd139 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:22.490322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba1807-bef8-459a-8dd3-c1ad704cd139 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:22.490591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba1807-bef8-459a-8dd3-c1ad704cd139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:22.510947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-94ba1807-bef8-459a-8dd3-c1ad704cd139 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:22.512132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-94ba1807-bef8-459a-8dd3-c1ad704cd139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:22.519851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1162/2363] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:22.524523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eece09-1068-4304-afdf-ca44684cac70 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:22.534198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eece09-1068-4304-afdf-ca44684cac70 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:22.539709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20143eb0-f9aa-403b-b3e3-f4dee6db59d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:22.540312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20143eb0-f9aa-403b-b3e3-f4dee6db59d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:22.540592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20143eb0-f9aa-403b-b3e3-f4dee6db59d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:22.540868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20143eb0-f9aa-403b-b3e3-f4dee6db59d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:22.541554 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20143eb0-f9aa-403b-b3e3-f4dee6db59d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:34:22.541864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20143eb0-f9aa-403b-b3e3-f4dee6db59d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:22.543492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20143eb0-f9aa-403b-b3e3-f4dee6db59d2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:22.556502 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1202/2364] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:34:22.566574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba1807-bef8-459a-8dd3-c1ad704cd139 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:22.576163 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba1807-bef8-459a-8dd3-c1ad704cd139 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:22.581076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308890a6-e1ae-4e18-b5a2-c2fdc7611497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:22.581699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308890a6-e1ae-4e18-b5a2-c2fdc7611497 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:22.581980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308890a6-e1ae-4e18-b5a2-c2fdc7611497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:22.582251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308890a6-e1ae-4e18-b5a2-c2fdc7611497 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:22.583397 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308890a6-e1ae-4e18-b5a2-c2fdc7611497 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:22.585295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308890a6-e1ae-4e18-b5a2-c2fdc7611497 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:22.702184 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1163/2365] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:34:22 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:34:22.705014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20143eb0-f9aa-403b-b3e3-f4dee6db59d2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:22.716881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20143eb0-f9aa-403b-b3e3-f4dee6db59d2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:22.778468 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bd039-8c57-4eb7-9cd7-8ad646f693fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:22.779075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bd039-8c57-4eb7-9cd7-8ad646f693fc admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:22.779363 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bd039-8c57-4eb7-9cd7-8ad646f693fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:22.779638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bd039-8c57-4eb7-9cd7-8ad646f693fc admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:22.780293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bd039-8c57-4eb7-9cd7-8ad646f693fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:22.782222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bd039-8c57-4eb7-9cd7-8ad646f693fc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:22.853198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1203/2366] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:34:22 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:34:22.857290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308890a6-e1ae-4e18-b5a2-c2fdc7611497 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:22.868409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308890a6-e1ae-4e18-b5a2-c2fdc7611497 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:22.873140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11aa6d-4e87-479b-b8cf-1f8f65813906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:22.873791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11aa6d-4e87-479b-b8cf-1f8f65813906 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:22.874085 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11aa6d-4e87-479b-b8cf-1f8f65813906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:22.874354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11aa6d-4e87-479b-b8cf-1f8f65813906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:22.874995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11aa6d-4e87-479b-b8cf-1f8f65813906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:22.876612 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11aa6d-4e87-479b-b8cf-1f8f65813906 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:22.887555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1164/2367] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:22.892337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bd039-8c57-4eb7-9cd7-8ad646f693fc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:22.902312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bd039-8c57-4eb7-9cd7-8ad646f693fc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:22.906927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5f34b-5952-462f-a5ca-4a898581a061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:22.907519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5f34b-5952-462f-a5ca-4a898581a061 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:22.907800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5f34b-5952-462f-a5ca-4a898581a061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:22.908074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5f34b-5952-462f-a5ca-4a898581a061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:22.908720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5f34b-5952-462f-a5ca-4a898581a061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:22.910344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5f34b-5952-462f-a5ca-4a898581a061 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:22.921261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1204/2368] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:22.925352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11aa6d-4e87-479b-b8cf-1f8f65813906 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:22.936459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11aa6d-4e87-479b-b8cf-1f8f65813906 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:22.941113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c969814-deaf-4c9d-96be-154e0163b0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:22.941784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c969814-deaf-4c9d-96be-154e0163b0e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:22.942062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c969814-deaf-4c9d-96be-154e0163b0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:22.942350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c969814-deaf-4c9d-96be-154e0163b0e3 admin admin] PATH_INFO: `/v3/projects/1ecfcd99992a4d84bb4b39ad987434a2/users/0a97218c760343af82bd2ec52c518ba7/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:22.956582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c969814-deaf-4c9d-96be-154e0163b0e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ecfcd99992a4d84bb4b39ad987434a2, user_id=0a97218c760343af82bd2ec52c518ba7, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:22.959579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c969814-deaf-4c9d-96be-154e0163b0e3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:23.024809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1165/2369] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:22 2022] PUT /identity/v3/projects/1ecfcd99992a4d84bb4b39ad987434a2/users/0a97218c760343af82bd2ec52c518ba7/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:23.036476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92545b8e-740e-4e2b-bdf3-d800b4317533 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:23.036790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92545b8e-740e-4e2b-bdf3-d800b4317533 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:23.037065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92545b8e-740e-4e2b-bdf3-d800b4317533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:23.058100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:23.065051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-92545b8e-740e-4e2b-bdf3-d800b4317533 None None] MFA Rules not processed for user `0a97218c760343af82bd2ec52c518ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:23.066351 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-92545b8e-740e-4e2b-bdf3-d800b4317533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:23.132401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1205/2370] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:23.143442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c969814-deaf-4c9d-96be-154e0163b0e3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:23.145127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6c969814-deaf-4c9d-96be-154e0163b0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:23.166824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:23.203513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c969814-deaf-4c9d-96be-154e0163b0e3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:23.207171 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c5e53-26f4-4830-bc3e-8b8f64009b7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:23.207703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5e53-26f4-4830-bc3e-8b8f64009b7f service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:23.208036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5e53-26f4-4830-bc3e-8b8f64009b7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:23.208277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5e53-26f4-4830-bc3e-8b8f64009b7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:23.216394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c5e53-26f4-4830-bc3e-8b8f64009b7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:23.219238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c5e53-26f4-4830-bc3e-8b8f64009b7f service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:23.227081 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1166/2371] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:23.251834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92545b8e-740e-4e2b-bdf3-d800b4317533 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:23.316200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92545b8e-740e-4e2b-bdf3-d800b4317533 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:23.321538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:23.322169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:23.322451 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:23.322726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] PATH_INFO: `/v3/users/0a97218c760343af82bd2ec52c518ba7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:23.323855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a97218c760343af82bd2ec52c518ba7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:23.325810 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:23.504422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a97218c760343af82bd2ec52c518ba7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:23.517089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a97218c760343af82bd2ec52c518ba7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:23.524256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a97218c760343af82bd2ec52c518ba7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:23.583970 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a97218c760343af82bd2ec52c518ba7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:23.584616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a97218c760343af82bd2ec52c518ba7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:23.590952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1206/2372] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:23 2022] DELETE /identity/v3/users/0a97218c760343af82bd2ec52c518ba7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:23.595475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05795-8c46-4dea-b67f-4a96a61b7982 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:23.596039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05795-8c46-4dea-b67f-4a96a61b7982 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:23.596039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05795-8c46-4dea-b67f-4a96a61b7982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:23.615727 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-17c05795-8c46-4dea-b67f-4a96a61b7982 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:23.617277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-17c05795-8c46-4dea-b67f-4a96a61b7982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:23.679328 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1167/2373] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:23.688974 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:23.690395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:23.712536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:23.749151 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1faf983-f7bd-4737-8ed4-ffd9f7983162 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:23.752813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92339d81-cfb2-4f11-aa22-0b65a92792b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:23.753415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92339d81-cfb2-4f11-aa22-0b65a92792b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:23.766129 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92339d81-cfb2-4f11-aa22-0b65a92792b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:23.766129 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92339d81-cfb2-4f11-aa22-0b65a92792b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:23.766129 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92339d81-cfb2-4f11-aa22-0b65a92792b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:23.766129 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92339d81-cfb2-4f11-aa22-0b65a92792b1 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:23.774176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1207/2374] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:24.157479 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c05795-8c46-4dea-b67f-4a96a61b7982 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:24.158961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-17c05795-8c46-4dea-b67f-4a96a61b7982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:24.171871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c05795-8c46-4dea-b67f-4a96a61b7982 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:24.176608 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:24.177240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:24.177542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:24.177807 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] PATH_INFO: `/v3/projects/1ecfcd99992a4d84bb4b39ad987434a2` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:24.179219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ecfcd99992a4d84bb4b39ad987434a2)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:24.181146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:24.279968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ecfcd99992a4d84bb4b39ad987434a2', 'request_id': 'req-f3c0b327-6bb1-4d94-a8f8-e56649e12929'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:24.336707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ecfcd99992a4d84bb4b39ad987434a2', 'request_id': 'req-f3c0b327-6bb1-4d94-a8f8-e56649e12929'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:24.343527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ecfcd99992a4d84bb4b39ad987434a2', 'request_id': 'req-f3c0b327-6bb1-4d94-a8f8-e56649e12929'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:24.353980 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] The token cache is being invalidate because project 1ecfcd99992a4d84bb4b39ad987434a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:24.354267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:24.354908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3c0b327-6bb1-4d94-a8f8-e56649e12929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:24.356368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1168/2375] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:24 2022] DELETE /identity/v3/projects/1ecfcd99992a4d84bb4b39ad987434a2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:24.361336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0cbc5-f477-4419-8600-4c06eeaffa44 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:24.361651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0cbc5-f477-4419-8600-4c06eeaffa44 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:24.361920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0cbc5-f477-4419-8600-4c06eeaffa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:24.381578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2fd0cbc5-f477-4419-8600-4c06eeaffa44 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:24.382765 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2fd0cbc5-f477-4419-8600-4c06eeaffa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:24.441090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1208/2376] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:24.449511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469be30-0c57-4a22-a9a7-f5597329bbee None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:24.449794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469be30-0c57-4a22-a9a7-f5597329bbee None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:24.450061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469be30-0c57-4a22-a9a7-f5597329bbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:24.471061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5469be30-0c57-4a22-a9a7-f5597329bbee None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:24.472289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5469be30-0c57-4a22-a9a7-f5597329bbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:24.480791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1169/2377] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:24.484952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0cbc5-f477-4419-8600-4c06eeaffa44 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:24.495289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0cbc5-f477-4419-8600-4c06eeaffa44 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:24.499776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39086abc-d393-4f0d-938f-0a51abb970bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:24.500370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39086abc-d393-4f0d-938f-0a51abb970bf admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:24.500651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39086abc-d393-4f0d-938f-0a51abb970bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:24.500923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39086abc-d393-4f0d-938f-0a51abb970bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:24.501610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39086abc-d393-4f0d-938f-0a51abb970bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:34:24.501917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39086abc-d393-4f0d-938f-0a51abb970bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:24.503500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39086abc-d393-4f0d-938f-0a51abb970bf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:24.514939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1209/2378] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:34:24.518966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469be30-0c57-4a22-a9a7-f5597329bbee None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:24.528339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469be30-0c57-4a22-a9a7-f5597329bbee None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:24.532938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e47f4-39eb-4a92-89a8-59dd5645ab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:24.533488 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e47f4-39eb-4a92-89a8-59dd5645ab3e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:24.533774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e47f4-39eb-4a92-89a8-59dd5645ab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:24.534045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e47f4-39eb-4a92-89a8-59dd5645ab3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:24.534672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e47f4-39eb-4a92-89a8-59dd5645ab3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:24.537210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e47f4-39eb-4a92-89a8-59dd5645ab3e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:24.618117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1170/2379] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:34:24 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:24.622521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39086abc-d393-4f0d-938f-0a51abb970bf admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:24.633194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39086abc-d393-4f0d-938f-0a51abb970bf admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:24.689908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944bbf4-7c9b-4660-adf7-4c2511e4b4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:24.690526 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944bbf4-7c9b-4660-adf7-4c2511e4b4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:24.690804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944bbf4-7c9b-4660-adf7-4c2511e4b4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:24.691077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944bbf4-7c9b-4660-adf7-4c2511e4b4b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:24.691773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944bbf4-7c9b-4660-adf7-4c2511e4b4b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:24.694308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944bbf4-7c9b-4660-adf7-4c2511e4b4b4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:24.790436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1210/2380] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:34:24 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:34:24.794484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e47f4-39eb-4a92-89a8-59dd5645ab3e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:24.806112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e47f4-39eb-4a92-89a8-59dd5645ab3e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:24.810750 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb2e71-9502-4755-91a6-fc4c95977601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:24.811342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb2e71-9502-4755-91a6-fc4c95977601 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:24.811622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb2e71-9502-4755-91a6-fc4c95977601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:24.811897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb2e71-9502-4755-91a6-fc4c95977601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:24.812569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb2e71-9502-4755-91a6-fc4c95977601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:24.814199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb2e71-9502-4755-91a6-fc4c95977601 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:24.825335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1171/2381] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:24.829206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944bbf4-7c9b-4660-adf7-4c2511e4b4b4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:24.839336 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944bbf4-7c9b-4660-adf7-4c2511e4b4b4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:24.843865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80e2e3-edea-4508-828c-c3ef5a35f756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:24.844456 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e2e3-edea-4508-828c-c3ef5a35f756 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:24.844734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e2e3-edea-4508-828c-c3ef5a35f756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:24.845006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e2e3-edea-4508-828c-c3ef5a35f756 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:24.845674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80e2e3-edea-4508-828c-c3ef5a35f756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:24.847261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80e2e3-edea-4508-828c-c3ef5a35f756 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:24.858353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1211/2382] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:34:24.862510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb2e71-9502-4755-91a6-fc4c95977601 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:24.869684 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:24.873340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb2e71-9502-4755-91a6-fc4c95977601 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:24.877955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e281-09fc-4464-9e7f-4aca7a8edac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:24.878595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e281-09fc-4464-9e7f-4aca7a8edac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:24.882593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e281-09fc-4464-9e7f-4aca7a8edac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:24.882593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e281-09fc-4464-9e7f-4aca7a8edac3 admin admin] PATH_INFO: `/v3/projects/9fb5b12fc7e045bc96a33630d5088740/users/a4e8ad7c8b4b4c9d83c0456b777e9e87/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:24.894697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16e281-09fc-4464-9e7f-4aca7a8edac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb5b12fc7e045bc96a33630d5088740, user_id=a4e8ad7c8b4b4c9d83c0456b777e9e87, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:24.897514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16e281-09fc-4464-9e7f-4aca7a8edac3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:24.958963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1172/2383] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:34:24 2022] PUT /identity/v3/projects/9fb5b12fc7e045bc96a33630d5088740/users/a4e8ad7c8b4b4c9d83c0456b777e9e87/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:24.977862 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af79d1f-c665-4ec6-87ee-543e98640adf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:24.978147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af79d1f-c665-4ec6-87ee-543e98640adf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:24.978423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af79d1f-c665-4ec6-87ee-543e98640adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:25.007006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2af79d1f-c665-4ec6-87ee-543e98640adf None None] MFA Rules not processed for user `a4e8ad7c8b4b4c9d83c0456b777e9e87`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:25.008215 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2af79d1f-c665-4ec6-87ee-543e98640adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:25.024481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:25.074791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1212/2384] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:25.086326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e281-09fc-4464-9e7f-4aca7a8edac3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:25.088036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0d16e281-09fc-4464-9e7f-4aca7a8edac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:25.145787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e281-09fc-4464-9e7f-4aca7a8edac3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:25.149567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856723e-000c-45d7-9be1-1b6b3884c02d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:25.150164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856723e-000c-45d7-9be1-1b6b3884c02d service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:25.150438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856723e-000c-45d7-9be1-1b6b3884c02d service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:25.150711 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856723e-000c-45d7-9be1-1b6b3884c02d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:25.158871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856723e-000c-45d7-9be1-1b6b3884c02d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:25.160289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856723e-000c-45d7-9be1-1b6b3884c02d service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:25.169945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1173/2385] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:30.092904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814a34-a6af-49f0-864c-1cb436d42af8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:30.093411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814a34-a6af-49f0-864c-1cb436d42af8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:30.093528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39814a34-a6af-49f0-864c-1cb436d42af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:30.113332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-39814a34-a6af-49f0-864c-1cb436d42af8 None None] MFA Rules not processed for user `a4e8ad7c8b4b4c9d83c0456b777e9e87`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:30.114956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-39814a34-a6af-49f0-864c-1cb436d42af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:30.121987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1213/2386] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:32.646076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856723e-000c-45d7-9be1-1b6b3884c02d service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:32.648377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8856723e-000c-45d7-9be1-1b6b3884c02d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:32.717381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856723e-000c-45d7-9be1-1b6b3884c02d service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:32.722081 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:32.722877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:32.723218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:32.723492 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] PATH_INFO: `/v3/users/b0a80d355be84cb7946c5dd2737af2ab` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:32.725310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a80d355be84cb7946c5dd2737af2ab)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:32.727315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:32.968317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a80d355be84cb7946c5dd2737af2ab'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:32.968960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a80d355be84cb7946c5dd2737af2ab'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:33.046950 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a80d355be84cb7946c5dd2737af2ab'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:33.061047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a80d355be84cb7946c5dd2737af2ab'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:33.068050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a80d355be84cb7946c5dd2737af2ab'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:33.074546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1174/2387] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:32 2022] DELETE /identity/v3/users/b0a80d355be84cb7946c5dd2737af2ab => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:33.085563 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908fa03-300c-4fd8-bd46-6002fd6ab0f0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:33.086199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908fa03-300c-4fd8-bd46-6002fd6ab0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:33.086199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908fa03-300c-4fd8-bd46-6002fd6ab0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:33.106289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2908fa03-300c-4fd8-bd46-6002fd6ab0f0 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:33.107529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2908fa03-300c-4fd8-bd46-6002fd6ab0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:33.170034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1214/2388] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:33.180366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:33.181965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:33.187559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:33.244105 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca387aa-cf3a-47c2-bab8-a0f7eff7eaa0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:33.248212 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd464f-61ba-4a36-990e-b875fd9b8106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:33.250553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd464f-61ba-4a36-990e-b875fd9b8106 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:33.250553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd464f-61ba-4a36-990e-b875fd9b8106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:33.250553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd464f-61ba-4a36-990e-b875fd9b8106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:33.260073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd464f-61ba-4a36-990e-b875fd9b8106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:33.261627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd464f-61ba-4a36-990e-b875fd9b8106 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:33.272451 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1175/2389] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:34:33.686496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908fa03-300c-4fd8-bd46-6002fd6ab0f0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:33.688148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2908fa03-300c-4fd8-bd46-6002fd6ab0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:33.701314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908fa03-300c-4fd8-bd46-6002fd6ab0f0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:33.706223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:33.711250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:33.711250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:33.711250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] PATH_INFO: `/v3/projects/fa2b6fc441fa4222a6b9ac498c4910c7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:33.711250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2b6fc441fa4222a6b9ac498c4910c7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:33.711745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:33.856754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2b6fc441fa4222a6b9ac498c4910c7', 'request_id': 'req-8297bdea-ddb4-4c7a-bf28-874c45da6a85'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:33.950361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2b6fc441fa4222a6b9ac498c4910c7', 'request_id': 'req-8297bdea-ddb4-4c7a-bf28-874c45da6a85'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:33.956852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2b6fc441fa4222a6b9ac498c4910c7', 'request_id': 'req-8297bdea-ddb4-4c7a-bf28-874c45da6a85'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:33.968564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] The token cache is being invalidate because project fa2b6fc441fa4222a6b9ac498c4910c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:34:33.968855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:33.969566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:33.971366 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1215/2390] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:34:33 2022] DELETE /identity/v3/projects/fa2b6fc441fa4222a6b9ac498c4910c7 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:43.330666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=25 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:43.965353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:53.565442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8200a-0fed-4373-be8e-14a08109d25f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:53.566008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8200a-0fed-4373-be8e-14a08109d25f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:53.566008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8200a-0fed-4373-be8e-14a08109d25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:53.586573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-35d8200a-0fed-4373-be8e-14a08109d25f None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:53.587761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-35d8200a-0fed-4373-be8e-14a08109d25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:53.646560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1176/2391] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:34:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:53.693867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:53.695571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:53.775850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297bdea-ddb4-4c7a-bf28-874c45da6a85 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:53.782042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb0c98-49b8-4c83-9dc5-1d22775ad679 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:53.783281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb0c98-49b8-4c83-9dc5-1d22775ad679 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:53.783566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb0c98-49b8-4c83-9dc5-1d22775ad679 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:53.783845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb0c98-49b8-4c83-9dc5-1d22775ad679 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:53.794369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb0c98-49b8-4c83-9dc5-1d22775ad679 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:53.795958 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb0c98-49b8-4c83-9dc5-1d22775ad679 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:53.804890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1216/2392] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:34:54.277224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84471a4-fce9-48ee-8740-06490a5b3d6e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:54.277549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84471a4-fce9-48ee-8740-06490a5b3d6e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:54.277830 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84471a4-fce9-48ee-8740-06490a5b3d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:54.298283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b84471a4-fce9-48ee-8740-06490a5b3d6e None None] MFA Rules not processed for user `b9db0f3de10842eb97dc6fc580f4489a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:54.299526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b84471a4-fce9-48ee-8740-06490a5b3d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:54.361075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1177/2393] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:54.686917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61ef61-c235-478a-a671-ef995116229d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:54.687304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61ef61-c235-478a-a671-ef995116229d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:54.687605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61ef61-c235-478a-a671-ef995116229d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:54.707032 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-df61ef61-c235-478a-a671-ef995116229d None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:54.708235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-df61ef61-c235-478a-a671-ef995116229d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:54.718463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1217/2394] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 1 05:34:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:54.736002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84471a4-fce9-48ee-8740-06490a5b3d6e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:54.749406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84471a4-fce9-48ee-8740-06490a5b3d6e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:54.754484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0d937-a764-4fc9-8302-03c9bdfd62c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:54.755088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0d937-a764-4fc9-8302-03c9bdfd62c3 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:54.755407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0d937-a764-4fc9-8302-03c9bdfd62c3 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:54.755981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0d937-a764-4fc9-8302-03c9bdfd62c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:54.764922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0d937-a764-4fc9-8302-03c9bdfd62c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:54.766569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0d937-a764-4fc9-8302-03c9bdfd62c3 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:54.776029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1178/2395] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:34:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:34:55.478991 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73583b25-5afe-4c66-8838-38a438049d45 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:55.479446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73583b25-5afe-4c66-8838-38a438049d45 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:55.479897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73583b25-5afe-4c66-8838-38a438049d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:55.503589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-73583b25-5afe-4c66-8838-38a438049d45 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:55.504921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-73583b25-5afe-4c66-8838-38a438049d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:55.513580 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1218/2396] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:34:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:55.527999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0d937-a764-4fc9-8302-03c9bdfd62c3 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:55.541034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0d937-a764-4fc9-8302-03c9bdfd62c3 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:55.547066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a1490-3510-4cc9-bc38-905332aa0de6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:55.551012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a1490-3510-4cc9-bc38-905332aa0de6 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:55.551012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a1490-3510-4cc9-bc38-905332aa0de6 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:55.551012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a1490-3510-4cc9-bc38-905332aa0de6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:55.559760 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a1490-3510-4cc9-bc38-905332aa0de6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:55.561509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a1490-3510-4cc9-bc38-905332aa0de6 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:55.571430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1179/2397] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:34:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:34:56.320602 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fad201-c228-42a6-a1f6-6bfe1c726a51 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:56.321109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fad201-c228-42a6-a1f6-6bfe1c726a51 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:56.321208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fad201-c228-42a6-a1f6-6bfe1c726a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:56.341305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-50fad201-c228-42a6-a1f6-6bfe1c726a51 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:56.342632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-50fad201-c228-42a6-a1f6-6bfe1c726a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:56.350310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1219/2398] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 1 05:34:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:56.362390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a1490-3510-4cc9-bc38-905332aa0de6 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:56.373916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a1490-3510-4cc9-bc38-905332aa0de6 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:56.378977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0060f-f45b-4889-8eb0-ad43f88cc693 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:56.379571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0060f-f45b-4889-8eb0-ad43f88cc693 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:56.379866 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0060f-f45b-4889-8eb0-ad43f88cc693 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:56.380176 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0060f-f45b-4889-8eb0-ad43f88cc693 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:56.389171 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0060f-f45b-4889-8eb0-ad43f88cc693 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:56.390798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0060f-f45b-4889-8eb0-ad43f88cc693 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:56.399625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1180/2399] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:34:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:34:59.152976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fad201-c228-42a6-a1f6-6bfe1c726a51 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:59.155665 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-50fad201-c228-42a6-a1f6-6bfe1c726a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:59.237282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fad201-c228-42a6-a1f6-6bfe1c726a51 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:59.242476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:59.243269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:59.243737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:59.244014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] PATH_INFO: `/v3/users/a4e8ad7c8b4b4c9d83c0456b777e9e87` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:59.245399 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e8ad7c8b4b4c9d83c0456b777e9e87)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:59.247593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:59.437195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e8ad7c8b4b4c9d83c0456b777e9e87'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:59.450677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e8ad7c8b4b4c9d83c0456b777e9e87'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:59.456500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:59.459299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e8ad7c8b4b4c9d83c0456b777e9e87'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:59.536785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e8ad7c8b4b4c9d83c0456b777e9e87'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:59.538209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e8ad7c8b4b4c9d83c0456b777e9e87'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:34:59.548006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1220/2400] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:34:59 2022] DELETE /identity/v3/users/a4e8ad7c8b4b4c9d83c0456b777e9e87 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:34:59.554511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337acd5e-2581-43cf-9778-2f6773b78587 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:59.558749 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337acd5e-2581-43cf-9778-2f6773b78587 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:59.558749 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337acd5e-2581-43cf-9778-2f6773b78587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:59.585331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-337acd5e-2581-43cf-9778-2f6773b78587 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:59.586899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-337acd5e-2581-43cf-9778-2f6773b78587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:59.665523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1181/2401] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:59.675527 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:59.677034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:59.777316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d59d0f-83bf-4c29-94ed-f690409dab2b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:59.781681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4110117-3032-40da-b155-b9a99146a3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:59.782444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4110117-3032-40da-b155-b9a99146a3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:59.782793 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4110117-3032-40da-b155-b9a99146a3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:59.783136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4110117-3032-40da-b155-b9a99146a3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:59.794635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4110117-3032-40da-b155-b9a99146a3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:59.796623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4110117-3032-40da-b155-b9a99146a3c0 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:59.809065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1221/2402] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:35:00.106622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337acd5e-2581-43cf-9778-2f6773b78587 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:00.108220 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-337acd5e-2581-43cf-9778-2f6773b78587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:00.131646 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337acd5e-2581-43cf-9778-2f6773b78587 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:00.147887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:00.148833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:00.149123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:00.149873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] PATH_INFO: `/v3/projects/9fb5b12fc7e045bc96a33630d5088740` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:00.151074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb5b12fc7e045bc96a33630d5088740)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:00.153142 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:00.160204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:00.325707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb5b12fc7e045bc96a33630d5088740', 'request_id': 'req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:00.384127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb5b12fc7e045bc96a33630d5088740', 'request_id': 'req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:00.391241 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb5b12fc7e045bc96a33630d5088740', 'request_id': 'req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:00.404659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] The token cache is being invalidate because project 9fb5b12fc7e045bc96a33630d5088740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:35:00.404941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:00.405636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:00.407267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1182/2403] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:35:00 2022] DELETE /identity/v3/projects/9fb5b12fc7e045bc96a33630d5088740 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:35:09.803051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=29,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:10.402964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:41.047755 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e5440-2697-4e65-9a33-17a35411bfad None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:41.048290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e5440-2697-4e65-9a33-17a35411bfad None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:41.048290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e5440-2697-4e65-9a33-17a35411bfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:41.067508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-614e5440-2697-4e65-9a33-17a35411bfad None None] MFA Rules not processed for user `b2c0f79be80341a483f02509d774007f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:35:41.068691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-614e5440-2697-4e65-9a33-17a35411bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:41.123185 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1222/2404] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:35:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:35:41.161145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:41.162642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:41.220451 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481871e7-94e9-4f5d-96ea-2017b8e6a6f7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:41.224041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f3c33-cbd8-4585-ab08-75e421e5b5ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c23597e0da2454abd2a5354d779b092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:41.224643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f3c33-cbd8-4585-ab08-75e421e5b5ef service glance] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:41.224922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f3c33-cbd8-4585-ab08-75e421e5b5ef service glance] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:41.225195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f3c33-cbd8-4585-ab08-75e421e5b5ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:41.226926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-109f3c33-cbd8-4585-ab08-75e421e5b5ef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:41.295830 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f3c33-cbd8-4585-ab08-75e421e5b5ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:41.297186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f3c33-cbd8-4585-ab08-75e421e5b5ef service glance] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:41.308893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1183/2405] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:41.911226 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e5440-2697-4e65-9a33-17a35411bfad None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:41.913492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-614e5440-2697-4e65-9a33-17a35411bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:41.974524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e5440-2697-4e65-9a33-17a35411bfad None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:41.978195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33690e-8925-4072-b220-6cb1219a95a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:41.978826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33690e-8925-4072-b220-6cb1219a95a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:41.979142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33690e-8925-4072-b220-6cb1219a95a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:41.979548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33690e-8925-4072-b220-6cb1219a95a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:41.981547 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9d33690e-8925-4072-b220-6cb1219a95a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:41.996238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33690e-8925-4072-b220-6cb1219a95a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:41.997874 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33690e-8925-4072-b220-6cb1219a95a4 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:42.009297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1223/2406] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:51.303673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=54 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:52.004792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=51 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:19.442021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f3c33-cbd8-4585-ab08-75e421e5b5ef service glance] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:19.443445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-109f3c33-cbd8-4585-ab08-75e421e5b5ef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:19.457998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f3c33-cbd8-4585-ab08-75e421e5b5ef service glance] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:19.463412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ca306-ed0e-40cc-bb67-73577572ce3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:19.463997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ca306-ed0e-40cc-bb67-73577572ce3c service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:19.464274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ca306-ed0e-40cc-bb67-73577572ce3c service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:19.464546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ca306-ed0e-40cc-bb67-73577572ce3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:19.466541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-227ca306-ed0e-40cc-bb67-73577572ce3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:19.524643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ca306-ed0e-40cc-bb67-73577572ce3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:19.526286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ca306-ed0e-40cc-bb67-73577572ce3c service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:19.535554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1184/2407] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:29.449013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33690e-8925-4072-b220-6cb1219a95a4 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:29.458474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33690e-8925-4072-b220-6cb1219a95a4 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:29.462192 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b498b3e-d745-4e92-ae31-57dd6cbf4b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:29.462796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b498b3e-d745-4e92-ae31-57dd6cbf4b5a service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:29.463075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b498b3e-d745-4e92-ae31-57dd6cbf4b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:29.463349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b498b3e-d745-4e92-ae31-57dd6cbf4b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:29.465068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7b498b3e-d745-4e92-ae31-57dd6cbf4b5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:29.475635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b498b3e-d745-4e92-ae31-57dd6cbf4b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:29.477060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b498b3e-d745-4e92-ae31-57dd6cbf4b5a service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:29.486623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1224/2408] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:29.531372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=28 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:33.506026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ca306-ed0e-40cc-bb67-73577572ce3c service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:33.507428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-227ca306-ed0e-40cc-bb67-73577572ce3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:33.521818 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ca306-ed0e-40cc-bb67-73577572ce3c service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:33.527372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77be65-931c-4248-9b7e-00b8d5a65218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:33.527966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77be65-931c-4248-9b7e-00b8d5a65218 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:33.528242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77be65-931c-4248-9b7e-00b8d5a65218 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:33.528511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77be65-931c-4248-9b7e-00b8d5a65218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:33.530297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bc77be65-931c-4248-9b7e-00b8d5a65218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:33.542422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77be65-931c-4248-9b7e-00b8d5a65218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:33.543989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77be65-931c-4248-9b7e-00b8d5a65218 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:33.555224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1185/2409] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:39.481654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:43.550831 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=6 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:48.164431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b498b3e-d745-4e92-ae31-57dd6cbf4b5a service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:48.178133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b498b3e-d745-4e92-ae31-57dd6cbf4b5a service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:48.183935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431715b-9995-4787-afd2-7b53e5df83e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:48.184423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431715b-9995-4787-afd2-7b53e5df83e5 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:48.184705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431715b-9995-4787-afd2-7b53e5df83e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:48.185234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431715b-9995-4787-afd2-7b53e5df83e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:48.187640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d431715b-9995-4787-afd2-7b53e5df83e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:48.247884 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431715b-9995-4787-afd2-7b53e5df83e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:48.250492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431715b-9995-4787-afd2-7b53e5df83e5 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:48.259930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1225/2410] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:55.681136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809901e5-0295-4c50-9595-817ab8d9de0e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:55.681709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809901e5-0295-4c50-9595-817ab8d9de0e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:55.681709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809901e5-0295-4c50-9595-817ab8d9de0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:55.703196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-809901e5-0295-4c50-9595-817ab8d9de0e None None] MFA Rules not processed for user `fb50b15e44814c6b82b2a75e3c74de0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:55.704474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-809901e5-0295-4c50-9595-817ab8d9de0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:55.711838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1186/2411] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:36:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:58.255334 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=28 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:05.696616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:06.026538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431715b-9995-4787-afd2-7b53e5df83e5 service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:06.027995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d431715b-9995-4787-afd2-7b53e5df83e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:06.094076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431715b-9995-4787-afd2-7b53e5df83e5 service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:06.098717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:06.099337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.099617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.099889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] PATH_INFO: `/v3/users/b2c0f79be80341a483f02509d774007f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.101004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c0f79be80341a483f02509d774007f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:06.102955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:06.269695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c0f79be80341a483f02509d774007f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:06.282371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c0f79be80341a483f02509d774007f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:06.289538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c0f79be80341a483f02509d774007f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:06.364585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c0f79be80341a483f02509d774007f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:06.365230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c0f79be80341a483f02509d774007f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:06.371406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1226/2412] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:37:06 2022] DELETE /identity/v3/users/b2c0f79be80341a483f02509d774007f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:06.601850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809901e5-0295-4c50-9595-817ab8d9de0e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:06.613577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809901e5-0295-4c50-9595-817ab8d9de0e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:06.672029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:06.672599 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.672879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.673155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] PATH_INFO: `/v3/projects/10dd440ea6f54e0497536117ccba4c9e` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.674324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10dd440ea6f54e0497536117ccba4c9e)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:06.676254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:06.783693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10dd440ea6f54e0497536117ccba4c9e', 'request_id': 'req-af3c7a59-37f9-4f99-9913-9fd3c7a64043'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:06.856040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10dd440ea6f54e0497536117ccba4c9e', 'request_id': 'req-af3c7a59-37f9-4f99-9913-9fd3c7a64043'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:06.862898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10dd440ea6f54e0497536117ccba4c9e', 'request_id': 'req-af3c7a59-37f9-4f99-9913-9fd3c7a64043'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:06.874045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] The token cache is being invalidate because project 10dd440ea6f54e0497536117ccba4c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:37:06.874473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:06.875100 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-af3c7a59-37f9-4f99-9913-9fd3c7a64043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:06.876584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1187/2413] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:37:06 2022] DELETE /identity/v3/projects/10dd440ea6f54e0497536117ccba4c9e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:16.367855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:16.870346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:22.136772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:22.138267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:22.213402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59bc17-4ae8-4536-b7e1-de0f0b0dc929 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:22.275691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41f102-92e8-4ec1-afea-fe1027a4d9dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:22.276297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41f102-92e8-4ec1-afea-fe1027a4d9dc service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:22.276579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41f102-92e8-4ec1-afea-fe1027a4d9dc service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:22.276869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41f102-92e8-4ec1-afea-fe1027a4d9dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:22.278659 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1d41f102-92e8-4ec1-afea-fe1027a4d9dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:22.337239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41f102-92e8-4ec1-afea-fe1027a4d9dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:22.338922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41f102-92e8-4ec1-afea-fe1027a4d9dc service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:22.347831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1227/2414] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:32.343610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:39.371811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7083345-4c6f-493c-b1b5-69b8358ec766 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:39.372321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7083345-4c6f-493c-b1b5-69b8358ec766 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:39.372321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7083345-4c6f-493c-b1b5-69b8358ec766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:39.392133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c7083345-4c6f-493c-b1b5-69b8358ec766 None None] MFA Rules not processed for user `b9db0f3de10842eb97dc6fc580f4489a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:39.393336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c7083345-4c6f-493c-b1b5-69b8358ec766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:39.400635 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1188/2415] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:37:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:39.705509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41f102-92e8-4ec1-afea-fe1027a4d9dc service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:39.706920 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1d41f102-92e8-4ec1-afea-fe1027a4d9dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:39.763813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41f102-92e8-4ec1-afea-fe1027a4d9dc service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:39.767360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf8bf4-0674-4fe1-8312-8fc6f07580f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:39.767964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8bf4-0674-4fe1-8312-8fc6f07580f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:39.768240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8bf4-0674-4fe1-8312-8fc6f07580f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:39.768512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8bf4-0674-4fe1-8312-8fc6f07580f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:39.770286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e7cf8bf4-0674-4fe1-8312-8fc6f07580f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:39.779569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf8bf4-0674-4fe1-8312-8fc6f07580f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:39.780976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf8bf4-0674-4fe1-8312-8fc6f07580f3 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:39.789535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1228/2416] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:40.227937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7083345-4c6f-493c-b1b5-69b8358ec766 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:40.229499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c7083345-4c6f-493c-b1b5-69b8358ec766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:40.300288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7083345-4c6f-493c-b1b5-69b8358ec766 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:40.305518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb54b7d-b892-4fb8-983f-55782c39c13a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:40.306128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb54b7d-b892-4fb8-983f-55782c39c13a service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:40.306408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb54b7d-b892-4fb8-983f-55782c39c13a service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:40.306679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb54b7d-b892-4fb8-983f-55782c39c13a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:40.308394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3fb54b7d-b892-4fb8-983f-55782c39c13a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:40.320427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb54b7d-b892-4fb8-983f-55782c39c13a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:40.322039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb54b7d-b892-4fb8-983f-55782c39c13a service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:40.331734 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1189/2417] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:49.785429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=26 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:50.327602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=35 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:51.460928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf8bf4-0674-4fe1-8312-8fc6f07580f3 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:51.462373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e7cf8bf4-0674-4fe1-8312-8fc6f07580f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:51.478827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf8bf4-0674-4fe1-8312-8fc6f07580f3 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:51.484217 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcaa194-22d3-4f59-b652-ca677e07d20b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:51.484887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaa194-22d3-4f59-b652-ca677e07d20b service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:51.485164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaa194-22d3-4f59-b652-ca677e07d20b service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:51.485439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaa194-22d3-4f59-b652-ca677e07d20b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:51.487250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bbcaa194-22d3-4f59-b652-ca677e07d20b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:51.499932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcaa194-22d3-4f59-b652-ca677e07d20b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:51.501566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcaa194-22d3-4f59-b652-ca677e07d20b service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:51.514686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1229/2418] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:01.510355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:18.927461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb54b7d-b892-4fb8-983f-55782c39c13a service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:18.937604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb54b7d-b892-4fb8-983f-55782c39c13a service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:18.941764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dbc66-e26e-4891-bf5d-3810f0304695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:18.942368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dbc66-e26e-4891-bf5d-3810f0304695 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:18.942644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dbc66-e26e-4891-bf5d-3810f0304695 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:18.942937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dbc66-e26e-4891-bf5d-3810f0304695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:18.944662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5f6dbc66-e26e-4891-bf5d-3810f0304695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:19.009023 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6dbc66-e26e-4891-bf5d-3810f0304695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:19.010414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6dbc66-e26e-4891-bf5d-3810f0304695 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:19.020007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1190/2419] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:22.545294 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcaa194-22d3-4f59-b652-ca677e07d20b service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:22.546967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bbcaa194-22d3-4f59-b652-ca677e07d20b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:22.564314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcaa194-22d3-4f59-b652-ca677e07d20b service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:22.568644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db9758-141e-4398-aa88-55a617602428 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:22.569246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db9758-141e-4398-aa88-55a617602428 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:22.569555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db9758-141e-4398-aa88-55a617602428 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:22.569826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db9758-141e-4398-aa88-55a617602428 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:22.571925 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-35db9758-141e-4398-aa88-55a617602428 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:22.584758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db9758-141e-4398-aa88-55a617602428 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:22.587421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db9758-141e-4398-aa88-55a617602428 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:22.597623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1230/2420] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:22.677609 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dbc66-e26e-4891-bf5d-3810f0304695 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:22.679487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5f6dbc66-e26e-4891-bf5d-3810f0304695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:22.694897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dbc66-e26e-4891-bf5d-3810f0304695 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:22.700475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:22.701211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:22.701582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:22.701927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] PATH_INFO: `/v3/users/fb50b15e44814c6b82b2a75e3c74de0a` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:22.703161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb50b15e44814c6b82b2a75e3c74de0a)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:22.705235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:22.908060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb50b15e44814c6b82b2a75e3c74de0a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:22.908656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb50b15e44814c6b82b2a75e3c74de0a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:22.984030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb50b15e44814c6b82b2a75e3c74de0a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:23.001547 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb50b15e44814c6b82b2a75e3c74de0a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:23.010961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ec3cb4e6-236e-4280-8c73-658b55fba7f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb50b15e44814c6b82b2a75e3c74de0a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:23.017489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1191/2421] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:38:22 2022] DELETE /identity/v3/users/fb50b15e44814c6b82b2a75e3c74de0a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:23.339953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db9758-141e-4398-aa88-55a617602428 service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:23.355355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db9758-141e-4398-aa88-55a617602428 service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:23.420035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:23.420598 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:23.420883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:23.421158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] PATH_INFO: `/v3/projects/cdbdacacad0f48c6a9238643e67ac6eb` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:23.422319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdbdacacad0f48c6a9238643e67ac6eb)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:23.428416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:23.579745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdbdacacad0f48c6a9238643e67ac6eb', 'request_id': 'req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:23.652688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdbdacacad0f48c6a9238643e67ac6eb', 'request_id': 'req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:23.660202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdbdacacad0f48c6a9238643e67ac6eb', 'request_id': 'req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:23.676071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] The token cache is being invalidate because project cdbdacacad0f48c6a9238643e67ac6eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:38:23.676369 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:23.677376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a4c6e3c1-c494-471d-83ed-00cb6a7cc3f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:23.678983 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1231/2422] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:38:23 2022] DELETE /identity/v3/projects/cdbdacacad0f48c6a9238643e67ac6eb => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:23.684919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d34495-2edf-48f6-aa12-344fbdcc4e91 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:23.685359 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d34495-2edf-48f6-aa12-344fbdcc4e91 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:23.685587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d34495-2edf-48f6-aa12-344fbdcc4e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:23.707857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c6d34495-2edf-48f6-aa12-344fbdcc4e91 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:23.709094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c6d34495-2edf-48f6-aa12-344fbdcc4e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:23.786685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1192/2423] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:23.795303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5e058-b0fa-44c6-ac8a-fd9c5de1004c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:23.795588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5e058-b0fa-44c6-ac8a-fd9c5de1004c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:23.795865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5e058-b0fa-44c6-ac8a-fd9c5de1004c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:23.816746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e7b5e058-b0fa-44c6-ac8a-fd9c5de1004c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:23.817989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e7b5e058-b0fa-44c6-ac8a-fd9c5de1004c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:23.826098 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1232/2424] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:23.830356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d34495-2edf-48f6-aa12-344fbdcc4e91 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:23.840253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d34495-2edf-48f6-aa12-344fbdcc4e91 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:23.844929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293e060-37f3-47a4-a203-d7a9fa1544e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:23.845554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e060-37f3-47a4-a203-d7a9fa1544e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:23.845835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e060-37f3-47a4-a203-d7a9fa1544e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:23.846100 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e060-37f3-47a4-a203-d7a9fa1544e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:23.846747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293e060-37f3-47a4-a203-d7a9fa1544e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:38:23.847047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293e060-37f3-47a4-a203-d7a9fa1544e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:23.848662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293e060-37f3-47a4-a203-d7a9fa1544e3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:23.850284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:23.862015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1193/2425] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:38:23.866710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5e058-b0fa-44c6-ac8a-fd9c5de1004c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:23.877317 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5e058-b0fa-44c6-ac8a-fd9c5de1004c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:23.882783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfae797-a90a-4a8d-b29e-d20bb2918083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:23.883723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfae797-a90a-4a8d-b29e-d20bb2918083 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:23.884006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfae797-a90a-4a8d-b29e-d20bb2918083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:23.884282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfae797-a90a-4a8d-b29e-d20bb2918083 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:23.884943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfae797-a90a-4a8d-b29e-d20bb2918083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:23.886953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfae797-a90a-4a8d-b29e-d20bb2918083 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:23.987508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1233/2426] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:38:23 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:23.990921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293e060-37f3-47a4-a203-d7a9fa1544e3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.002943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293e060-37f3-47a4-a203-d7a9fa1544e3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.076697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79cf64-0aab-4e0e-847a-cad89d87f2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.077510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79cf64-0aab-4e0e-847a-cad89d87f2de admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.077978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79cf64-0aab-4e0e-847a-cad89d87f2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.078436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79cf64-0aab-4e0e-847a-cad89d87f2de admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.079187 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79cf64-0aab-4e0e-847a-cad89d87f2de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.083572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79cf64-0aab-4e0e-847a-cad89d87f2de admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.177541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1194/2427] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:38:23 2022] POST /identity/v3/users => generated 295 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:24.179091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfae797-a90a-4a8d-b29e-d20bb2918083 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.190766 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfae797-a90a-4a8d-b29e-d20bb2918083 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.195990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d57977-6497-4a2b-9368-f52142669d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.196485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d57977-6497-4a2b-9368-f52142669d78 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.196836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d57977-6497-4a2b-9368-f52142669d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.197174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d57977-6497-4a2b-9368-f52142669d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.197941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d57977-6497-4a2b-9368-f52142669d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.200242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d57977-6497-4a2b-9368-f52142669d78 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.213219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1234/2428] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:24.216462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79cf64-0aab-4e0e-847a-cad89d87f2de admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.228410 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79cf64-0aab-4e0e-847a-cad89d87f2de admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.233421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817030e8-cfb5-4bba-bd6c-cd12697bbb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.234119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817030e8-cfb5-4bba-bd6c-cd12697bbb5f admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.234554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817030e8-cfb5-4bba-bd6c-cd12697bbb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.234894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817030e8-cfb5-4bba-bd6c-cd12697bbb5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.235599 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817030e8-cfb5-4bba-bd6c-cd12697bbb5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.237266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817030e8-cfb5-4bba-bd6c-cd12697bbb5f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.249077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1195/2429] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:24.253345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d57977-6497-4a2b-9368-f52142669d78 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.263867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d57977-6497-4a2b-9368-f52142669d78 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.269053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce059db-20ee-44df-bfa9-36bea07876e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.269053 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce059db-20ee-44df-bfa9-36bea07876e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.269364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce059db-20ee-44df-bfa9-36bea07876e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.269644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce059db-20ee-44df-bfa9-36bea07876e8 admin admin] PATH_INFO: `/v3/projects/459dd5f92ff94f8b81d1e7213e8e7170/users/1aa49a620d0e42b0947215d200fa45a1/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.284883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce059db-20ee-44df-bfa9-36bea07876e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459dd5f92ff94f8b81d1e7213e8e7170, user_id=1aa49a620d0e42b0947215d200fa45a1, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.287681 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce059db-20ee-44df-bfa9-36bea07876e8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.407379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1235/2430] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:24 2022] PUT /identity/v3/projects/459dd5f92ff94f8b81d1e7213e8e7170/users/1aa49a620d0e42b0947215d200fa45a1/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:24.410937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2888e4f-ac94-4c92-92e1-0a3c3eeb6e10 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.411224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2888e4f-ac94-4c92-92e1-0a3c3eeb6e10 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.411501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2888e4f-ac94-4c92-92e1-0a3c3eeb6e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.431902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f2888e4f-ac94-4c92-92e1-0a3c3eeb6e10 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:24.435527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f2888e4f-ac94-4c92-92e1-0a3c3eeb6e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:24.493151 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1196/2431] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:24.502696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce059db-20ee-44df-bfa9-36bea07876e8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.504401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9ce059db-20ee-44df-bfa9-36bea07876e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:24.518603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:24.563064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce059db-20ee-44df-bfa9-36bea07876e8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.566963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4dd35-db79-4304-95ac-915b4681929b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.567560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4dd35-db79-4304-95ac-915b4681929b service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.567835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4dd35-db79-4304-95ac-915b4681929b service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.568101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4dd35-db79-4304-95ac-915b4681929b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.577054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4dd35-db79-4304-95ac-915b4681929b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.578493 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4dd35-db79-4304-95ac-915b4681929b service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.588047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1236/2432] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:28.865944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6aa58-158a-49cd-ae20-d49735410a95 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:28.866578 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6aa58-158a-49cd-ae20-d49735410a95 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:28.866774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6aa58-158a-49cd-ae20-d49735410a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:28.886552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-67d6aa58-158a-49cd-ae20-d49735410a95 None None] MFA Rules not processed for user `e9ad5076ef93462fbe614d7fc323935d`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:28.887755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-67d6aa58-158a-49cd-ae20-d49735410a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:28.946679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1197/2433] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:38:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:30.049239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f0d62-6d6e-4691-bd1c-6beb9eca9405 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:30.049856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f0d62-6d6e-4691-bd1c-6beb9eca9405 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:30.050013 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f0d62-6d6e-4691-bd1c-6beb9eca9405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:30.084969 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bf6f0d62-6d6e-4691-bd1c-6beb9eca9405 None None] MFA Rules not processed for user `1aa49a620d0e42b0947215d200fa45a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:30.087556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bf6f0d62-6d6e-4691-bd1c-6beb9eca9405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:30.173094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1237/2434] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:38:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:38:30.193508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6aa58-158a-49cd-ae20-d49735410a95 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:30.199427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-67d6aa58-158a-49cd-ae20-d49735410a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:30.223227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:30.286133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6aa58-158a-49cd-ae20-d49735410a95 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:30.293574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13612cb-cb4d-48d6-908a-0eead2845b1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:30.294926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13612cb-cb4d-48d6-908a-0eead2845b1d service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:30.295480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13612cb-cb4d-48d6-908a-0eead2845b1d service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:30.295693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13612cb-cb4d-48d6-908a-0eead2845b1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:30.306022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13612cb-cb4d-48d6-908a-0eead2845b1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:30.307617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13612cb-cb4d-48d6-908a-0eead2845b1d service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:30.318166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1198/2435] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:33.036630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dccad-a08d-4136-93ea-cb3bbc3339fe None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:33.036630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dccad-a08d-4136-93ea-cb3bbc3339fe None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:33.036630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dccad-a08d-4136-93ea-cb3bbc3339fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:33.069424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f95dccad-a08d-4136-93ea-cb3bbc3339fe None None] MFA Rules not processed for user `1aa49a620d0e42b0947215d200fa45a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:33.071432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f95dccad-a08d-4136-93ea-cb3bbc3339fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:33.079872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1238/2436] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:38:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:33.703789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13612cb-cb4d-48d6-908a-0eead2845b1d service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:33.719863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13612cb-cb4d-48d6-908a-0eead2845b1d service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:33.728082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecebce2c-b818-4dea-a3f9-7e19c09a0a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:33.728756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebce2c-b818-4dea-a3f9-7e19c09a0a7e service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:33.729032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebce2c-b818-4dea-a3f9-7e19c09a0a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:33.729304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebce2c-b818-4dea-a3f9-7e19c09a0a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:33.731457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ecebce2c-b818-4dea-a3f9-7e19c09a0a7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:33.742628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecebce2c-b818-4dea-a3f9-7e19c09a0a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:33.744368 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecebce2c-b818-4dea-a3f9-7e19c09a0a7e service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:33.753779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1199/2437] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:43.063312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=55 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:43.749823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=32 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:39:28.996958 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dccad-a08d-4136-93ea-cb3bbc3339fe None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:28.998380 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f95dccad-a08d-4136-93ea-cb3bbc3339fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:29.142218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dccad-a08d-4136-93ea-cb3bbc3339fe None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:29.147211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402ffcd-6129-4b63-abe9-183625bdf62c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:29.147815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ffcd-6129-4b63-abe9-183625bdf62c service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:29.149378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ffcd-6129-4b63-abe9-183625bdf62c service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:29.150007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ffcd-6129-4b63-abe9-183625bdf62c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:29.159341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e402ffcd-6129-4b63-abe9-183625bdf62c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:29.172094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402ffcd-6129-4b63-abe9-183625bdf62c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:29.173887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402ffcd-6129-4b63-abe9-183625bdf62c service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:29.184307 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1239/2438] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:39:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:39:33.295493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fbb2a-adc4-4ba5-9518-1c6e4c827455 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:33.296017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fbb2a-adc4-4ba5-9518-1c6e4c827455 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:33.296017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fbb2a-adc4-4ba5-9518-1c6e4c827455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:33.315115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b55fbb2a-adc4-4ba5-9518-1c6e4c827455 None None] MFA Rules not processed for user `b9db0f3de10842eb97dc6fc580f4489a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:39:33.316569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b55fbb2a-adc4-4ba5-9518-1c6e4c827455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:33.371588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1200/2439] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:39:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:39:39.179882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=63 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:39:43.365322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=25 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:29.248759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c596f8-04fd-458e-8115-3c2f04393a3b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:29.249253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c596f8-04fd-458e-8115-3c2f04393a3b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:29.249348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c596f8-04fd-458e-8115-3c2f04393a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:29.279183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-05c596f8-04fd-458e-8115-3c2f04393a3b None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:29.280371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-05c596f8-04fd-458e-8115-3c2f04393a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:29.343137 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1240/2440] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:40:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:29.490678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fbb2a-adc4-4ba5-9518-1c6e4c827455 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:29.492204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b55fbb2a-adc4-4ba5-9518-1c6e4c827455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:29.562986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fbb2a-adc4-4ba5-9518-1c6e4c827455 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:29.568448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e31dc5-6691-47cc-96cc-34b8d9b05435 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:29.569049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e31dc5-6691-47cc-96cc-34b8d9b05435 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:29.569326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e31dc5-6691-47cc-96cc-34b8d9b05435 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:29.569639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e31dc5-6691-47cc-96cc-34b8d9b05435 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:29.577613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e31dc5-6691-47cc-96cc-34b8d9b05435 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:29.579224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e31dc5-6691-47cc-96cc-34b8d9b05435 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:29.587963 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1201/2441] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:40:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:40:34.763178 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c596f8-04fd-458e-8115-3c2f04393a3b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:34.764761 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-05c596f8-04fd-458e-8115-3c2f04393a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:34.819332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c596f8-04fd-458e-8115-3c2f04393a3b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:34.824560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c1911-7b7a-46a2-a98f-9c0cd9e0540a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:34.825532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c1911-7b7a-46a2-a98f-9c0cd9e0540a service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:34.825817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c1911-7b7a-46a2-a98f-9c0cd9e0540a service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:34.826092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c1911-7b7a-46a2-a98f-9c0cd9e0540a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:34.827851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-981c1911-7b7a-46a2-a98f-9c0cd9e0540a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:34.904045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c1911-7b7a-46a2-a98f-9c0cd9e0540a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:34.905756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c1911-7b7a-46a2-a98f-9c0cd9e0540a service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:34.933520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1241/2442] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:39.584982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=32 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:39.975006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e31dc5-6691-47cc-96cc-34b8d9b05435 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:39.977125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-98e31dc5-6691-47cc-96cc-34b8d9b05435 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:39.992870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e31dc5-6691-47cc-96cc-34b8d9b05435 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:39.997391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:39.998070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:39.998374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:39.998702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] PATH_INFO: `/v3/users/e9ad5076ef93462fbe614d7fc323935d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:40.014326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ad5076ef93462fbe614d7fc323935d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:40.018315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:40.228257 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ad5076ef93462fbe614d7fc323935d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:40.229314 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ad5076ef93462fbe614d7fc323935d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:40.310313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ad5076ef93462fbe614d7fc323935d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:40.327660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ad5076ef93462fbe614d7fc323935d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:40.335772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ad5076ef93462fbe614d7fc323935d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:40.342714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1202/2443] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:39 2022] DELETE /identity/v3/users/e9ad5076ef93462fbe614d7fc323935d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:40.638653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c1911-7b7a-46a2-a98f-9c0cd9e0540a service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:40.655965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c1911-7b7a-46a2-a98f-9c0cd9e0540a service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:40.723933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:40.723933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:40.723933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:40.723933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] PATH_INFO: `/v3/projects/337817b568b64e98bc2bfd177e984351` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:40.734345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] RBAC: Authorizing `identity:delete_project(project_id=337817b568b64e98bc2bfd177e984351)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:40.736414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:40.761541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:40.927531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '337817b568b64e98bc2bfd177e984351', 'request_id': 'req-b17c0604-68e5-46e1-b363-e9d4f606801f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:41.012908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '337817b568b64e98bc2bfd177e984351', 'request_id': 'req-b17c0604-68e5-46e1-b363-e9d4f606801f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:41.025686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '337817b568b64e98bc2bfd177e984351', 'request_id': 'req-b17c0604-68e5-46e1-b363-e9d4f606801f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:41.033883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] The token cache is being invalidate because project 337817b568b64e98bc2bfd177e984351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:41.034289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:41.035016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b17c0604-68e5-46e1-b363-e9d4f606801f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:41.040119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1242/2444] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:40 2022] DELETE /identity/v3/projects/337817b568b64e98bc2bfd177e984351 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:41.041157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.041698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0850468-aa4b-449d-8154-3d9748a6544a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.042109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0850468-aa4b-449d-8154-3d9748a6544a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.042459 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0850468-aa4b-449d-8154-3d9748a6544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.043688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:41.103182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a0850468-aa4b-449d-8154-3d9748a6544a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:41.104393 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a0850468-aa4b-449d-8154-3d9748a6544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:41.107147 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399abe62-1d47-4635-8aa8-eced63beece9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.163103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a1a47-5b62-4909-aad2-ba511072dc70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.163721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a1a47-5b62-4909-aad2-ba511072dc70 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.164000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a1a47-5b62-4909-aad2-ba511072dc70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.164269 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a1a47-5b62-4909-aad2-ba511072dc70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.166262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e53a1a47-5b62-4909-aad2-ba511072dc70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:41.169657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1243/2445] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:41.179169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d3471-ac76-4d38-abef-185c28f18ae7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.179715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d3471-ac76-4d38-abef-185c28f18ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.179995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d3471-ac76-4d38-abef-185c28f18ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.200756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fa1d3471-ac76-4d38-abef-185c28f18ae7 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:41.202143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fa1d3471-ac76-4d38-abef-185c28f18ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:41.210408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1244/2446] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:40:41.212149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:41.214556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d3471-ac76-4d38-abef-185c28f18ae7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.224412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d3471-ac76-4d38-abef-185c28f18ae7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.229325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a3d2a-68e2-4589-8b59-9823879e4915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.229988 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a3d2a-68e2-4589-8b59-9823879e4915 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.230263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a3d2a-68e2-4589-8b59-9823879e4915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.230530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a3d2a-68e2-4589-8b59-9823879e4915 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.231170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a3d2a-68e2-4589-8b59-9823879e4915 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:40:41.231469 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a3d2a-68e2-4589-8b59-9823879e4915 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.233115 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a3d2a-68e2-4589-8b59-9823879e4915 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.246901 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1245/2447] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:40:41.250529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a3d2a-68e2-4589-8b59-9823879e4915 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.260471 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a3d2a-68e2-4589-8b59-9823879e4915 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.265036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e703085-88ba-4862-adb1-71ec0fb93042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.265651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e703085-88ba-4862-adb1-71ec0fb93042 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.265928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e703085-88ba-4862-adb1-71ec0fb93042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.266194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a1a47-5b62-4909-aad2-ba511072dc70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.266311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e703085-88ba-4862-adb1-71ec0fb93042 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.266802 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e703085-88ba-4862-adb1-71ec0fb93042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.267924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a1a47-5b62-4909-aad2-ba511072dc70 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.268720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e703085-88ba-4862-adb1-71ec0fb93042 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.291427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1203/2448] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:40:41.367548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1246/2449] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:40:41.369070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a1a47-5b62-4909-aad2-ba511072dc70 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.380511 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a1a47-5b62-4909-aad2-ba511072dc70 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.443404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297a70b-5173-497e-8aa0-6653c446cd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.444007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297a70b-5173-497e-8aa0-6653c446cd41 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.444284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297a70b-5173-497e-8aa0-6653c446cd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.444552 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297a70b-5173-497e-8aa0-6653c446cd41 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.445165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297a70b-5173-497e-8aa0-6653c446cd41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.447119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297a70b-5173-497e-8aa0-6653c446cd41 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.537991 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1204/2450] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:41.542167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e703085-88ba-4862-adb1-71ec0fb93042 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.554382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e703085-88ba-4862-adb1-71ec0fb93042 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.559831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630791c-77b7-4f27-971c-e7a619959b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.560427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630791c-77b7-4f27-971c-e7a619959b48 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.560706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630791c-77b7-4f27-971c-e7a619959b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.560977 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630791c-77b7-4f27-971c-e7a619959b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.561642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630791c-77b7-4f27-971c-e7a619959b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.563229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630791c-77b7-4f27-971c-e7a619959b48 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.574120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1247/2451] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:41.578002 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297a70b-5173-497e-8aa0-6653c446cd41 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.588020 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297a70b-5173-497e-8aa0-6653c446cd41 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.592634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6030506-010b-431b-9a5e-a4d555b897b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.593223 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6030506-010b-431b-9a5e-a4d555b897b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.593534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6030506-010b-431b-9a5e-a4d555b897b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.593813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6030506-010b-431b-9a5e-a4d555b897b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.594451 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6030506-010b-431b-9a5e-a4d555b897b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.596037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6030506-010b-431b-9a5e-a4d555b897b7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.606068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1205/2452] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:41.610171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630791c-77b7-4f27-971c-e7a619959b48 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.621065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630791c-77b7-4f27-971c-e7a619959b48 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.625752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838feea-57cb-4c11-83ce-e25da8d23d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.626394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838feea-57cb-4c11-83ce-e25da8d23d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.626670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838feea-57cb-4c11-83ce-e25da8d23d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.626940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838feea-57cb-4c11-83ce-e25da8d23d68 admin admin] PATH_INFO: `/v3/projects/67627314140b46d394958ccb595eec9f/users/07ac2f8866724f2ca22b84726f8bae43/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.642450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838feea-57cb-4c11-83ce-e25da8d23d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67627314140b46d394958ccb595eec9f, user_id=07ac2f8866724f2ca22b84726f8bae43, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.645689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838feea-57cb-4c11-83ce-e25da8d23d68 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.647874 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:41.727565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1248/2453] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:41 2022] PUT /identity/v3/projects/67627314140b46d394958ccb595eec9f/users/07ac2f8866724f2ca22b84726f8bae43/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:41.738007 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac6a78-9919-4d0e-9f32-aa0ccdf36cb9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.738438 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac6a78-9919-4d0e-9f32-aa0ccdf36cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.738730 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac6a78-9919-4d0e-9f32-aa0ccdf36cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.767316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e1ac6a78-9919-4d0e-9f32-aa0ccdf36cb9 None None] MFA Rules not processed for user `07ac2f8866724f2ca22b84726f8bae43`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:41.768528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e1ac6a78-9919-4d0e-9f32-aa0ccdf36cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:41.836457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1206/2454] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:41.841706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838feea-57cb-4c11-83ce-e25da8d23d68 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.904553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838feea-57cb-4c11-83ce-e25da8d23d68 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.909186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a387fc-bc6b-40d9-b0f3-80ae5ea84165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.909813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a387fc-bc6b-40d9-b0f3-80ae5ea84165 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.910095 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a387fc-bc6b-40d9-b0f3-80ae5ea84165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.910429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a387fc-bc6b-40d9-b0f3-80ae5ea84165 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.911050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a387fc-bc6b-40d9-b0f3-80ae5ea84165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.914454 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a387fc-bc6b-40d9-b0f3-80ae5ea84165 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.032297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1249/2455] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:42.036895 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac6a78-9919-4d0e-9f32-aa0ccdf36cb9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.047638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac6a78-9919-4d0e-9f32-aa0ccdf36cb9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.051785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:42.105763 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97131-6a02-48b2-9457-29365e18e0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.106536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97131-6a02-48b2-9457-29365e18e0d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.106689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97131-6a02-48b2-9457-29365e18e0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.106956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97131-6a02-48b2-9457-29365e18e0d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.107574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f97131-6a02-48b2-9457-29365e18e0d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.109666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f97131-6a02-48b2-9457-29365e18e0d0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.219080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1207/2456] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:42 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:40:42.224367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a387fc-bc6b-40d9-b0f3-80ae5ea84165 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.234939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a387fc-bc6b-40d9-b0f3-80ae5ea84165 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.239924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773635d1-b5ac-41b2-bf53-a85e48c18f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.240638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773635d1-b5ac-41b2-bf53-a85e48c18f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.240912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773635d1-b5ac-41b2-bf53-a85e48c18f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.241180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773635d1-b5ac-41b2-bf53-a85e48c18f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.241888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773635d1-b5ac-41b2-bf53-a85e48c18f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.243476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773635d1-b5ac-41b2-bf53-a85e48c18f09 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.255494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1250/2457] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:42.261799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97131-6a02-48b2-9457-29365e18e0d0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.272779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97131-6a02-48b2-9457-29365e18e0d0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.277420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c0c80-9a54-4a6b-a3c9-a378814ab249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.278109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c0c80-9a54-4a6b-a3c9-a378814ab249 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.278385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c0c80-9a54-4a6b-a3c9-a378814ab249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.278655 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c0c80-9a54-4a6b-a3c9-a378814ab249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.279291 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c0c80-9a54-4a6b-a3c9-a378814ab249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.280869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c0c80-9a54-4a6b-a3c9-a378814ab249 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.291592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1208/2458] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:42.295844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773635d1-b5ac-41b2-bf53-a85e48c18f09 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.306162 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773635d1-b5ac-41b2-bf53-a85e48c18f09 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.310931 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4981c1a-1ae5-40b7-90aa-5435430bcdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.311610 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4981c1a-1ae5-40b7-90aa-5435430bcdc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.311895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4981c1a-1ae5-40b7-90aa-5435430bcdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.312164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4981c1a-1ae5-40b7-90aa-5435430bcdc2 admin admin] PATH_INFO: `/v3/projects/a4e059816114478eb9369c0a34a32854/users/bac26e4c62b84ab2ad8c7aec329cf4ff/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.326940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4981c1a-1ae5-40b7-90aa-5435430bcdc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e059816114478eb9369c0a34a32854, user_id=bac26e4c62b84ab2ad8c7aec329cf4ff, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.329637 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4981c1a-1ae5-40b7-90aa-5435430bcdc2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.396743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1251/2459] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:42 2022] PUT /identity/v3/projects/a4e059816114478eb9369c0a34a32854/users/bac26e4c62b84ab2ad8c7aec329cf4ff/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:42.407330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941042f2-ff0a-4c2f-bf9c-3157d7e25c54 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.407620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941042f2-ff0a-4c2f-bf9c-3157d7e25c54 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.407902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941042f2-ff0a-4c2f-bf9c-3157d7e25c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.439650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-941042f2-ff0a-4c2f-bf9c-3157d7e25c54 None None] MFA Rules not processed for user `bac26e4c62b84ab2ad8c7aec329cf4ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:42.441446 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-941042f2-ff0a-4c2f-bf9c-3157d7e25c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:42.487101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4981c1a-1ae5-40b7-90aa-5435430bcdc2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.489039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f4981c1a-1ae5-40b7-90aa-5435430bcdc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:42.514089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1209/2460] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:40:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:40:42.518516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941042f2-ff0a-4c2f-bf9c-3157d7e25c54 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.556351 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4981c1a-1ae5-40b7-90aa-5435430bcdc2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.561845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c81f2-d6e7-4234-988e-c982fdb6535b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.562646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c81f2-d6e7-4234-988e-c982fdb6535b service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.562993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c81f2-d6e7-4234-988e-c982fdb6535b service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.563331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c81f2-d6e7-4234-988e-c982fdb6535b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.565168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fd2c81f2-d6e7-4234-988e-c982fdb6535b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:42.600968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941042f2-ff0a-4c2f-bf9c-3157d7e25c54 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.606335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ae824-5ccf-431f-838e-9542ccf70642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.607604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ae824-5ccf-431f-838e-9542ccf70642 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.607604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ae824-5ccf-431f-838e-9542ccf70642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.607604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ae824-5ccf-431f-838e-9542ccf70642 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.608256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ae824-5ccf-431f-838e-9542ccf70642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.610372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ae824-5ccf-431f-838e-9542ccf70642 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.630962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:42.648082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c81f2-d6e7-4234-988e-c982fdb6535b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.649466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c81f2-d6e7-4234-988e-c982fdb6535b service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.661081 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1252/2461] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:42.740765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:42.747694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1210/2462] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:42 2022] POST /identity/v3/projects => generated 387 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:40:42.751791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c81f2-d6e7-4234-988e-c982fdb6535b service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.762310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c81f2-d6e7-4234-988e-c982fdb6535b service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.821799 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb546688-db68-4016-9b27-31cd2315a008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.822404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb546688-db68-4016-9b27-31cd2315a008 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.822703 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb546688-db68-4016-9b27-31cd2315a008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.822953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb546688-db68-4016-9b27-31cd2315a008 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.823582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb546688-db68-4016-9b27-31cd2315a008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.825503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb546688-db68-4016-9b27-31cd2315a008 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.916688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1253/2463] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:42 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:42.920748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ae824-5ccf-431f-838e-9542ccf70642 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.931484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ae824-5ccf-431f-838e-9542ccf70642 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.936062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a5ab3-3520-429b-98dd-95aa4deeaf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.936642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a5ab3-3520-429b-98dd-95aa4deeaf4f admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.936918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a5ab3-3520-429b-98dd-95aa4deeaf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.937186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a5ab3-3520-429b-98dd-95aa4deeaf4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.937888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a5ab3-3520-429b-98dd-95aa4deeaf4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.939478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a5ab3-3520-429b-98dd-95aa4deeaf4f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.949611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1211/2464] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:42.953636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb546688-db68-4016-9b27-31cd2315a008 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.964210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb546688-db68-4016-9b27-31cd2315a008 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.968753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548fdfe-8b99-4af3-89ea-174ede35cb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.969392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548fdfe-8b99-4af3-89ea-174ede35cb58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.969695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548fdfe-8b99-4af3-89ea-174ede35cb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.969963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548fdfe-8b99-4af3-89ea-174ede35cb58 admin admin] PATH_INFO: `/v3/projects/f73dfbc3acdf4e1e811eabd4925caac5/users/4587242cb32949749cb6692884ed510b/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.984803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548fdfe-8b99-4af3-89ea-174ede35cb58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f73dfbc3acdf4e1e811eabd4925caac5, user_id=4587242cb32949749cb6692884ed510b, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.987532 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548fdfe-8b99-4af3-89ea-174ede35cb58 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:43.087118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1254/2465] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:42 2022] PUT /identity/v3/projects/f73dfbc3acdf4e1e811eabd4925caac5/users/4587242cb32949749cb6692884ed510b/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:40:43.091107 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a5ab3-3520-429b-98dd-95aa4deeaf4f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:43.101687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a5ab3-3520-429b-98dd-95aa4deeaf4f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:43.157802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa9805-bb67-4106-b0fd-8581f7ef5946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:43.158404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa9805-bb67-4106-b0fd-8581f7ef5946 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:43.158680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa9805-bb67-4106-b0fd-8581f7ef5946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:43.158950 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa9805-bb67-4106-b0fd-8581f7ef5946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:43.159613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa9805-bb67-4106-b0fd-8581f7ef5946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:43.161829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa9805-bb67-4106-b0fd-8581f7ef5946 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:43.171546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1212/2466] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:43.175652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548fdfe-8b99-4af3-89ea-174ede35cb58 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:43.186529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548fdfe-8b99-4af3-89ea-174ede35cb58 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:43.191267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16a5c1-4222-4f2c-8ecc-8fe2fec87de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:43.192519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16a5c1-4222-4f2c-8ecc-8fe2fec87de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:43.193041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16a5c1-4222-4f2c-8ecc-8fe2fec87de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:43.193446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16a5c1-4222-4f2c-8ecc-8fe2fec87de5 admin admin] PATH_INFO: `/v3/projects/f73dfbc3acdf4e1e811eabd4925caac5/users/4587242cb32949749cb6692884ed510b/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:43.195437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16a5c1-4222-4f2c-8ecc-8fe2fec87de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f73dfbc3acdf4e1e811eabd4925caac5, user_id=4587242cb32949749cb6692884ed510b, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:43.198112 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16a5c1-4222-4f2c-8ecc-8fe2fec87de5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:43.208786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1255/2467] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:43 2022] PUT /identity/v3/projects/f73dfbc3acdf4e1e811eabd4925caac5/users/4587242cb32949749cb6692884ed510b/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:40:43.220140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca10d8a-989d-430d-82c4-9fb94f7bc52d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:43.220424 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca10d8a-989d-430d-82c4-9fb94f7bc52d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:43.220816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca10d8a-989d-430d-82c4-9fb94f7bc52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:43.248191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-aca10d8a-989d-430d-82c4-9fb94f7bc52d None None] MFA Rules not processed for user `4587242cb32949749cb6692884ed510b`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:43.249449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-aca10d8a-989d-430d-82c4-9fb94f7bc52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:43.315909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1213/2468] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:40:43 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:43.327853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16a5c1-4222-4f2c-8ecc-8fe2fec87de5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:43.329543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-af16a5c1-4222-4f2c-8ecc-8fe2fec87de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:43.409132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16a5c1-4222-4f2c-8ecc-8fe2fec87de5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:43.413391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd59a9-dd90-43aa-b6d6-3c1d8eef752c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:43.414017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd59a9-dd90-43aa-b6d6-3c1d8eef752c service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:43.414295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd59a9-dd90-43aa-b6d6-3c1d8eef752c service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:43.414568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd59a9-dd90-43aa-b6d6-3c1d8eef752c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:43.423432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd59a9-dd90-43aa-b6d6-3c1d8eef752c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:43.424794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd59a9-dd90-43aa-b6d6-3c1d8eef752c service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:43.434719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1256/2469] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:44.445015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca10d8a-989d-430d-82c4-9fb94f7bc52d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:44.446490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-aca10d8a-989d-430d-82c4-9fb94f7bc52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:44.459472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca10d8a-989d-430d-82c4-9fb94f7bc52d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:44.469611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb2dd0-2c3b-4bb9-a801-dc51fda9ac45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:44.470238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb2dd0-2c3b-4bb9-a801-dc51fda9ac45 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:44.470509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb2dd0-2c3b-4bb9-a801-dc51fda9ac45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:44.470772 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb2dd0-2c3b-4bb9-a801-dc51fda9ac45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:44.530746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb2dd0-2c3b-4bb9-a801-dc51fda9ac45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:44.532219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb2dd0-2c3b-4bb9-a801-dc51fda9ac45 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:44.542625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1214/2470] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:46.633297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83146338-52c1-41d3-89d9-fef07c9e33ce None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:46.633973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83146338-52c1-41d3-89d9-fef07c9e33ce None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:46.633973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83146338-52c1-41d3-89d9-fef07c9e33ce None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:46.635507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1257/2471] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:40:46.641421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a020c03-f809-40b9-8b45-3600e91b641e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:46.641910 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a020c03-f809-40b9-8b45-3600e91b641e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:46.643235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a020c03-f809-40b9-8b45-3600e91b641e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:46.660637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:46.683198 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4a020c03-f809-40b9-8b45-3600e91b641e None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:46.684430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4a020c03-f809-40b9-8b45-3600e91b641e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:46.762844 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1215/2472] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:40:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:46.774252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83146338-52c1-41d3-89d9-fef07c9e33ce None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:46.775940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-83146338-52c1-41d3-89d9-fef07c9e33ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:46.798798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:46.835731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83146338-52c1-41d3-89d9-fef07c9e33ce None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:46.839351 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a0383-4847-4ef3-8234-b8b15c43eb78 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c23597e0da2454abd2a5354d779b092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:46.839980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a0383-4847-4ef3-8234-b8b15c43eb78 service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:46.840267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a0383-4847-4ef3-8234-b8b15c43eb78 service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:46.840542 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a0383-4847-4ef3-8234-b8b15c43eb78 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:46.851105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a0383-4847-4ef3-8234-b8b15c43eb78 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:46.852578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a0383-4847-4ef3-8234-b8b15c43eb78 service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:46.863426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1258/2473] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:40:47.071214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef13214-1532-4c6d-b7f5-de94eb005f22 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:47.071789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef13214-1532-4c6d-b7f5-de94eb005f22 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:47.072068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef13214-1532-4c6d-b7f5-de94eb005f22 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:47.073480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1216/2474] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:40:47.079084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d5ddc-8f2a-49f4-a70f-6fb8624c3ed4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:47.079367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d5ddc-8f2a-49f4-a70f-6fb8624c3ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:47.079664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d5ddc-8f2a-49f4-a70f-6fb8624c3ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:47.099786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-109d5ddc-8f2a-49f4-a70f-6fb8624c3ed4 None None] MFA Rules not processed for user `0e21910fca40479ca04e7f0418f7dd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:47.101444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-109d5ddc-8f2a-49f4-a70f-6fb8624c3ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:47.151305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1259/2475] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:40:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:47.163896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef13214-1532-4c6d-b7f5-de94eb005f22 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:47.165604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6ef13214-1532-4c6d-b7f5-de94eb005f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:47.179696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef13214-1532-4c6d-b7f5-de94eb005f22 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:47.183285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838589f-e4b4-4a7e-8995-f017047f924d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:47.183877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838589f-e4b4-4a7e-8995-f017047f924d service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:47.184154 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838589f-e4b4-4a7e-8995-f017047f924d service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:47.184546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838589f-e4b4-4a7e-8995-f017047f924d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:47.192983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838589f-e4b4-4a7e-8995-f017047f924d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:47.194484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838589f-e4b4-4a7e-8995-f017047f924d service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:47.202687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1217/2476] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:48.295027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431afe-03aa-4fb5-99a1-de7519acf23a None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:48.295660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431afe-03aa-4fb5-99a1-de7519acf23a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:48.295660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431afe-03aa-4fb5-99a1-de7519acf23a None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:48.297050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1260/2477] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:40:48.301902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ebab-c9bb-4abb-a5e4-99bc5d241b96 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:48.302874 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ebab-c9bb-4abb-a5e4-99bc5d241b96 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:48.303037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ebab-c9bb-4abb-a5e4-99bc5d241b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:48.323517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d8d4ebab-c9bb-4abb-a5e4-99bc5d241b96 None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:48.324784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d8d4ebab-c9bb-4abb-a5e4-99bc5d241b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:48.334143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1218/2478] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:40:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:48.345136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89431afe-03aa-4fb5-99a1-de7519acf23a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:48.355552 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89431afe-03aa-4fb5-99a1-de7519acf23a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:48.359070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca40e-30c5-4aa0-81f5-750e02d12649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:48.359653 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca40e-30c5-4aa0-81f5-750e02d12649 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:48.359924 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca40e-30c5-4aa0-81f5-750e02d12649 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:48.360194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca40e-30c5-4aa0-81f5-750e02d12649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:48.371406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaca40e-30c5-4aa0-81f5-750e02d12649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:48.372865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaca40e-30c5-4aa0-81f5-750e02d12649 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:48.385036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1261/2479] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:49.353705 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4ebab-c9bb-4abb-a5e4-99bc5d241b96 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:49.363200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4ebab-c9bb-4abb-a5e4-99bc5d241b96 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:49.367310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a93a0-6ed4-4996-b5fb-35656e1dea39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:49.367899 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a93a0-6ed4-4996-b5fb-35656e1dea39 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:49.368180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a93a0-6ed4-4996-b5fb-35656e1dea39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:49.368453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a93a0-6ed4-4996-b5fb-35656e1dea39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:49.425362 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a93a0-6ed4-4996-b5fb-35656e1dea39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:49.426863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a93a0-6ed4-4996-b5fb-35656e1dea39 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:49.436258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1219/2480] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:53.221634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca40e-30c5-4aa0-81f5-750e02d12649 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:53.285789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca40e-30c5-4aa0-81f5-750e02d12649 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:53.290491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:53.291156 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:53.291437 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:53.291708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] PATH_INFO: `/v3/users/07ac2f8866724f2ca22b84726f8bae43` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:53.292811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ac2f8866724f2ca22b84726f8bae43)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:53.294747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:53.433110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ac2f8866724f2ca22b84726f8bae43'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:53.444189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:53.446487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ac2f8866724f2ca22b84726f8bae43'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:53.453311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ac2f8866724f2ca22b84726f8bae43'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:53.527326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ac2f8866724f2ca22b84726f8bae43'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:53.532416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ac2f8866724f2ca22b84726f8bae43'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:53.534584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1262/2481] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:53 2022] DELETE /identity/v3/users/07ac2f8866724f2ca22b84726f8bae43 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:53.539236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8a90a-89f9-41d5-b017-0deef5b3e727 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:53.539699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8a90a-89f9-41d5-b017-0deef5b3e727 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:53.540086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8a90a-89f9-41d5-b017-0deef5b3e727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:53.562265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-03a8a90a-89f9-41d5-b017-0deef5b3e727 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:53.563696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-03a8a90a-89f9-41d5-b017-0deef5b3e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:53.623754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1220/2482] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:53.634398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:53.635822 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:53.698463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482ac61-bf52-49eb-9ba5-09ddd6ce6af2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:53.702286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a738727-4347-4269-bc76-2942e2030ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:53.702904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a738727-4347-4269-bc76-2942e2030ade service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:53.703280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a738727-4347-4269-bc76-2942e2030ade service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:53.703462 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a738727-4347-4269-bc76-2942e2030ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:53.719579 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a738727-4347-4269-bc76-2942e2030ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:53.721056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a738727-4347-4269-bc76-2942e2030ade service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:53.732657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1263/2483] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:54.216614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8a90a-89f9-41d5-b017-0deef5b3e727 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:54.218622 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-03a8a90a-89f9-41d5-b017-0deef5b3e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:54.233005 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8a90a-89f9-41d5-b017-0deef5b3e727 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:54.238521 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:54.239193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:54.239619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:54.243983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] PATH_INFO: `/v3/projects/67627314140b46d394958ccb595eec9f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:54.243983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67627314140b46d394958ccb595eec9f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:54.243983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:54.245434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:54.363218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67627314140b46d394958ccb595eec9f', 'request_id': 'req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:54.429580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67627314140b46d394958ccb595eec9f', 'request_id': 'req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:54.436924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67627314140b46d394958ccb595eec9f', 'request_id': 'req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:54.449329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] The token cache is being invalidate because project 67627314140b46d394958ccb595eec9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:54.449656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:54.450338 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:54.451989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1221/2484] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:54 2022] DELETE /identity/v3/projects/67627314140b46d394958ccb595eec9f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:54.456048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a738727-4347-4269-bc76-2942e2030ade service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:54.466487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a738727-4347-4269-bc76-2942e2030ade service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:54.532630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:54.536068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:54.536068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:54.536068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] PATH_INFO: `/v3/users/bac26e4c62b84ab2ad8c7aec329cf4ff` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:54.536068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bac26e4c62b84ab2ad8c7aec329cf4ff)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:54.537848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:54.712005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bac26e4c62b84ab2ad8c7aec329cf4ff'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:54.727988 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bac26e4c62b84ab2ad8c7aec329cf4ff'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:54.734978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bac26e4c62b84ab2ad8c7aec329cf4ff'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:54.819235 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bac26e4c62b84ab2ad8c7aec329cf4ff'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:54.819955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bac26e4c62b84ab2ad8c7aec329cf4ff'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:54.823225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:54.832041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1264/2485] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:54 2022] DELETE /identity/v3/users/bac26e4c62b84ab2ad8c7aec329cf4ff => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:55.328795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:55.330926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:55.345115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f76aec-baf3-44ef-b62e-7c8e626c24e5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:55.404150 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:55.404784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:55.405062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:55.405331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] PATH_INFO: `/v3/projects/a4e059816114478eb9369c0a34a32854` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:55.406484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e059816114478eb9369c0a34a32854)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:55.408447 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:55.516862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e059816114478eb9369c0a34a32854', 'request_id': 'req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:55.590431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e059816114478eb9369c0a34a32854', 'request_id': 'req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:55.598256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e059816114478eb9369c0a34a32854', 'request_id': 'req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:55.612148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] The token cache is being invalidate because project a4e059816114478eb9369c0a34a32854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:55.613607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:55.613607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:55.614775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1222/2486] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:55 2022] DELETE /identity/v3/projects/a4e059816114478eb9369c0a34a32854 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:55.619903 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:55.621396 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:55.635176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fe3c3-9e1a-46c3-8ff9-80168bca6321 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:55.696288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:55.697003 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:55.697288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:55.697582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] PATH_INFO: `/v3/users/4587242cb32949749cb6692884ed510b` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:55.698711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4587242cb32949749cb6692884ed510b)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:55.700677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:55.856097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4587242cb32949749cb6692884ed510b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:55.869008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4587242cb32949749cb6692884ed510b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:55.876050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4587242cb32949749cb6692884ed510b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:55.927148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4587242cb32949749cb6692884ed510b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:55.928402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-45525cf3-b232-4a1b-b128-c15d1a68d313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4587242cb32949749cb6692884ed510b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:55.935099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1265/2487] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:55 2022] DELETE /identity/v3/users/4587242cb32949749cb6692884ed510b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:40:56.353214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:56.354782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:56.367658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575f9e0-8e91-4123-b5de-b8078ab5a9f4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:56.387066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:56.426155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:56.426777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:56.427051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:56.427320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] PATH_INFO: `/v3/projects/f73dfbc3acdf4e1e811eabd4925caac5` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:56.429115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f73dfbc3acdf4e1e811eabd4925caac5)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:56.431091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:56.509646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e492-2eb3-4a1a-8c98-15c10e274d77 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:56.509934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e492-2eb3-4a1a-8c98-15c10e274d77 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:56.510211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e492-2eb3-4a1a-8c98-15c10e274d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:56.529630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-29c7e492-2eb3-4a1a-8c98-15c10e274d77 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:56.530836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-29c7e492-2eb3-4a1a-8c98-15c10e274d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:56.557400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f73dfbc3acdf4e1e811eabd4925caac5', 'request_id': 'req-25ab96e9-d140-4170-86c7-6cf6bfacdad7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:56.593794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1266/2488] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:40:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:40:56.605846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7e492-2eb3-4a1a-8c98-15c10e274d77 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:56.607229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-29c7e492-2eb3-4a1a-8c98-15c10e274d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:56.635906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:56.641570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f73dfbc3acdf4e1e811eabd4925caac5', 'request_id': 'req-25ab96e9-d140-4170-86c7-6cf6bfacdad7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:56.648855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f73dfbc3acdf4e1e811eabd4925caac5', 'request_id': 'req-25ab96e9-d140-4170-86c7-6cf6bfacdad7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:56.660415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] The token cache is being invalidate because project f73dfbc3acdf4e1e811eabd4925caac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:56.660708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:56.661355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-25ab96e9-d140-4170-86c7-6cf6bfacdad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:56.663420 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1223/2489] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:56 2022] DELETE /identity/v3/projects/f73dfbc3acdf4e1e811eabd4925caac5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:56.668989 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783c23c-7d6c-4651-a528-532192c5101d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:56.669273 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783c23c-7d6c-4651-a528-532192c5101d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:56.671329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783c23c-7d6c-4651-a528-532192c5101d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:56.683811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7e492-2eb3-4a1a-8c98-15c10e274d77 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:56.690105 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b783c23c-7d6c-4651-a528-532192c5101d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:56.691306 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b783c23c-7d6c-4651-a528-532192c5101d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:56.749880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfce483-5f6a-4193-bcee-8f924d3eab6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:56.750495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfce483-5f6a-4193-bcee-8f924d3eab6a service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:56.751027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfce483-5f6a-4193-bcee-8f924d3eab6a service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:56.751027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfce483-5f6a-4193-bcee-8f924d3eab6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:56.752816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cbfce483-5f6a-4193-bcee-8f924d3eab6a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:56.762529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1224/2490] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:40:56.770996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18f04e-1028-407f-b4af-71e75fa98d47 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:56.771280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18f04e-1028-407f-b4af-71e75fa98d47 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:56.771598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18f04e-1028-407f-b4af-71e75fa98d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:56.791884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2d18f04e-1028-407f-b4af-71e75fa98d47 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:56.794385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2d18f04e-1028-407f-b4af-71e75fa98d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:56.801061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1225/2491] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:40:56.805262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18f04e-1028-407f-b4af-71e75fa98d47 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:56.811597 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfce483-5f6a-4193-bcee-8f924d3eab6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:56.813723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfce483-5f6a-4193-bcee-8f924d3eab6a service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:56.814744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18f04e-1028-407f-b4af-71e75fa98d47 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:56.820687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c400e-9676-4015-b76a-877134cd8584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:56.821270 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c400e-9676-4015-b76a-877134cd8584 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:56.821686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c400e-9676-4015-b76a-877134cd8584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:56.821957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c400e-9676-4015-b76a-877134cd8584 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:56.822589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c400e-9676-4015-b76a-877134cd8584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:40:56.822887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c400e-9676-4015-b76a-877134cd8584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:56.824067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1267/2492] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:40:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:40:56.824516 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c400e-9676-4015-b76a-877134cd8584 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:56.841359 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1226/2493] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:40:56.842572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfce483-5f6a-4193-bcee-8f924d3eab6a service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:56.853624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfce483-5f6a-4193-bcee-8f924d3eab6a service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:56.858651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccbdb9-9fe5-4df6-a7b4-068d5336403b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:56.859346 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccbdb9-9fe5-4df6-a7b4-068d5336403b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:56.859625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccbdb9-9fe5-4df6-a7b4-068d5336403b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:56.859897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccbdb9-9fe5-4df6-a7b4-068d5336403b admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:56.860713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccbdb9-9fe5-4df6-a7b4-068d5336403b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:56.863348 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccbdb9-9fe5-4df6-a7b4-068d5336403b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:56.939933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1268/2494] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:56 2022] POST /identity/v3/projects => generated 401 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:56.945029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c400e-9676-4015-b76a-877134cd8584 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:56.955188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c400e-9676-4015-b76a-877134cd8584 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:56.987920 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:57.027956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf6bac-0d79-43e7-8b76-6c18826a9e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:57.028870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf6bac-0d79-43e7-8b76-6c18826a9e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:57.029161 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf6bac-0d79-43e7-8b76-6c18826a9e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:57.029672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf6bac-0d79-43e7-8b76-6c18826a9e5d admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:57.030652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf6bac-0d79-43e7-8b76-6c18826a9e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:57.033051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf6bac-0d79-43e7-8b76-6c18826a9e5d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:57.155821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1227/2495] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:56 2022] POST /identity/v3/users => generated 301 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:57.161153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccbdb9-9fe5-4df6-a7b4-068d5336403b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:57.174714 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccbdb9-9fe5-4df6-a7b4-068d5336403b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:57.180907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac58e0-8bf4-4387-9bce-02e43ccd3a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:57.181826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac58e0-8bf4-4387-9bce-02e43ccd3a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:57.182297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac58e0-8bf4-4387-9bce-02e43ccd3a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:57.182649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac58e0-8bf4-4387-9bce-02e43ccd3a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:57.183400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac58e0-8bf4-4387-9bce-02e43ccd3a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:57.185101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac58e0-8bf4-4387-9bce-02e43ccd3a13 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:57.198565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1269/2496] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:40:57.203877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf6bac-0d79-43e7-8b76-6c18826a9e5d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:57.220847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf6bac-0d79-43e7-8b76-6c18826a9e5d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:57.227596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb4d75-99ce-4c90-aa48-356aae4ba0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:57.228398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb4d75-99ce-4c90-aa48-356aae4ba0ec admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:57.228774 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb4d75-99ce-4c90-aa48-356aae4ba0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:57.229145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb4d75-99ce-4c90-aa48-356aae4ba0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:57.230010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb4d75-99ce-4c90-aa48-356aae4ba0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:57.231909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb4d75-99ce-4c90-aa48-356aae4ba0ec admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:57.248332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1228/2497] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:40:57.252431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac58e0-8bf4-4387-9bce-02e43ccd3a13 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:57.264719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac58e0-8bf4-4387-9bce-02e43ccd3a13 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:57.269434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8ab16-ef42-4680-9024-cc27cdccec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:57.270291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8ab16-ef42-4680-9024-cc27cdccec1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:57.270575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8ab16-ef42-4680-9024-cc27cdccec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:57.270837 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8ab16-ef42-4680-9024-cc27cdccec1a admin admin] PATH_INFO: `/v3/projects/7cc3640de35b4364bc3ad8475b10d1e7/users/d9c36395091a4b1c8a9f5473a3a9bfbf/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:57.293644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8ab16-ef42-4680-9024-cc27cdccec1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc3640de35b4364bc3ad8475b10d1e7, user_id=d9c36395091a4b1c8a9f5473a3a9bfbf, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:57.296600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8ab16-ef42-4680-9024-cc27cdccec1a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:57.398588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1270/2498] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:57 2022] PUT /identity/v3/projects/7cc3640de35b4364bc3ad8475b10d1e7/users/d9c36395091a4b1c8a9f5473a3a9bfbf/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:40:57.414169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9f580-f5f1-4ac0-801b-775b17fe3017 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:57.414737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9f580-f5f1-4ac0-801b-775b17fe3017 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:57.414737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9f580-f5f1-4ac0-801b-775b17fe3017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:57.454878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-dbb9f580-f5f1-4ac0-801b-775b17fe3017 None None] MFA Rules not processed for user `d9c36395091a4b1c8a9f5473a3a9bfbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:57.456000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dbb9f580-f5f1-4ac0-801b-775b17fe3017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:57.535245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1229/2499] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:40:57 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:57.540148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8ab16-ef42-4680-9024-cc27cdccec1a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:57.554946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:57.629250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8ab16-ef42-4680-9024-cc27cdccec1a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:57.635222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb8268-f838-43dd-8e2f-12acda4b8f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:57.635834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb8268-f838-43dd-8e2f-12acda4b8f8b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:57.636110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb8268-f838-43dd-8e2f-12acda4b8f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:57.636722 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb8268-f838-43dd-8e2f-12acda4b8f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:57.636997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb8268-f838-43dd-8e2f-12acda4b8f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:57.638951 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb8268-f838-43dd-8e2f-12acda4b8f8b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:57.731130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1271/2500] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:57 2022] POST /identity/v3/projects => generated 399 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:57.734880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9f580-f5f1-4ac0-801b-775b17fe3017 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:57.748426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9f580-f5f1-4ac0-801b-775b17fe3017 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:57.944237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab0a54-427b-4e85-9f71-45c50dfb7815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:57.944837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0a54-427b-4e85-9f71-45c50dfb7815 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:57.945110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0a54-427b-4e85-9f71-45c50dfb7815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:57.945378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0a54-427b-4e85-9f71-45c50dfb7815 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:57.946033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab0a54-427b-4e85-9f71-45c50dfb7815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:57.947998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab0a54-427b-4e85-9f71-45c50dfb7815 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:58.060514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1230/2501] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:57 2022] POST /identity/v3/users => generated 306 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:58.064603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb8268-f838-43dd-8e2f-12acda4b8f8b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:58.076311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb8268-f838-43dd-8e2f-12acda4b8f8b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:58.081011 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d35f1-d71a-4e7c-9dfb-0a1939de698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:58.081661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d35f1-d71a-4e7c-9dfb-0a1939de698b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:58.081942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d35f1-d71a-4e7c-9dfb-0a1939de698b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:58.082217 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d35f1-d71a-4e7c-9dfb-0a1939de698b admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:58.082908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d35f1-d71a-4e7c-9dfb-0a1939de698b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:58.085351 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d35f1-d71a-4e7c-9dfb-0a1939de698b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:58.100811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1272/2502] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:40:58.101946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab0a54-427b-4e85-9f71-45c50dfb7815 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:58.116896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab0a54-427b-4e85-9f71-45c50dfb7815 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:58.121798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf20310-285e-4092-8adc-c2995b9251f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:58.122465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf20310-285e-4092-8adc-c2995b9251f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:58.122737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf20310-285e-4092-8adc-c2995b9251f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:58.123001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf20310-285e-4092-8adc-c2995b9251f7 admin admin] PATH_INFO: `/v3/projects/423a1a8972874e96a49c934be2c93c55/users/8760c253066d4a1d9c36f11eaa8db419/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:58.139423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf20310-285e-4092-8adc-c2995b9251f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423a1a8972874e96a49c934be2c93c55, user_id=8760c253066d4a1d9c36f11eaa8db419, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:58.142276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf20310-285e-4092-8adc-c2995b9251f7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:58.211857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1231/2503] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:58 2022] PUT /identity/v3/projects/423a1a8972874e96a49c934be2c93c55/users/8760c253066d4a1d9c36f11eaa8db419/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:40:58.213066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d35f1-d71a-4e7c-9dfb-0a1939de698b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:58.224205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d35f1-d71a-4e7c-9dfb-0a1939de698b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:58.287107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd22661-9daa-46d6-a801-5c034e83a4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:58.287919 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd22661-9daa-46d6-a801-5c034e83a4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:58.288198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd22661-9daa-46d6-a801-5c034e83a4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:58.288473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd22661-9daa-46d6-a801-5c034e83a4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:58.289117 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd22661-9daa-46d6-a801-5c034e83a4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:58.290847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd22661-9daa-46d6-a801-5c034e83a4b0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:58.305612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1273/2504] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:40:58.309679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf20310-285e-4092-8adc-c2995b9251f7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:58.316513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:58.321490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf20310-285e-4092-8adc-c2995b9251f7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:58.326644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e14413-0a8f-4ebb-ba3b-792523869dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:58.327306 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e14413-0a8f-4ebb-ba3b-792523869dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:58.327581 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e14413-0a8f-4ebb-ba3b-792523869dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:58.327846 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e14413-0a8f-4ebb-ba3b-792523869dd1 admin admin] PATH_INFO: `/v3/projects/423a1a8972874e96a49c934be2c93c55/users/8760c253066d4a1d9c36f11eaa8db419/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:58.329892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e14413-0a8f-4ebb-ba3b-792523869dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423a1a8972874e96a49c934be2c93c55, user_id=8760c253066d4a1d9c36f11eaa8db419, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:58.332629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e14413-0a8f-4ebb-ba3b-792523869dd1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:58.343840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1232/2505] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:58 2022] PUT /identity/v3/projects/423a1a8972874e96a49c934be2c93c55/users/8760c253066d4a1d9c36f11eaa8db419/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:58.355368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f86a3-3ffd-4380-ba7f-2932e5ab1de0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:58.356135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f86a3-3ffd-4380-ba7f-2932e5ab1de0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:58.356541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f86a3-3ffd-4380-ba7f-2932e5ab1de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:58.391145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6b7f86a3-3ffd-4380-ba7f-2932e5ab1de0 None None] MFA Rules not processed for user `8760c253066d4a1d9c36f11eaa8db419`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:58.392575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6b7f86a3-3ffd-4380-ba7f-2932e5ab1de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:58.476842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1274/2506] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:40:58 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:40:58.487678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e14413-0a8f-4ebb-ba3b-792523869dd1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:58.489462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c5e14413-0a8f-4ebb-ba3b-792523869dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:58.557680 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e14413-0a8f-4ebb-ba3b-792523869dd1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:58.562145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41766abe-acde-48b0-8a44-e2909d205977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:58.562779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41766abe-acde-48b0-8a44-e2909d205977 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:58.563040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41766abe-acde-48b0-8a44-e2909d205977 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:58.563302 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41766abe-acde-48b0-8a44-e2909d205977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:58.573872 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41766abe-acde-48b0-8a44-e2909d205977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:58.575228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41766abe-acde-48b0-8a44-e2909d205977 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:58.592217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1233/2507] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:58 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:58.818914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f86a3-3ffd-4380-ba7f-2932e5ab1de0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:58.833091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f86a3-3ffd-4380-ba7f-2932e5ab1de0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:58.837347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6ed97-d2c3-4a44-b1e7-482885e22a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:58.837999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6ed97-d2c3-4a44-b1e7-482885e22a70 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:58.838281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6ed97-d2c3-4a44-b1e7-482885e22a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:58.839281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6ed97-d2c3-4a44-b1e7-482885e22a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:58.843557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:58.910035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6ed97-d2c3-4a44-b1e7-482885e22a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:58.911656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6ed97-d2c3-4a44-b1e7-482885e22a70 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:58.921953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1275/2508] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:58 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:05.140750 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41766abe-acde-48b0-8a44-e2909d205977 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:05.231999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41766abe-acde-48b0-8a44-e2909d205977 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:05.236588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:05.237394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:05.237707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:05.237946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] PATH_INFO: `/v3/users/d9c36395091a4b1c8a9f5473a3a9bfbf` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:05.241514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9c36395091a4b1c8a9f5473a3a9bfbf)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:05.247191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:05.433892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9c36395091a4b1c8a9f5473a3a9bfbf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:05.434683 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9c36395091a4b1c8a9f5473a3a9bfbf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:05.503489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9c36395091a4b1c8a9f5473a3a9bfbf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:05.523751 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9c36395091a4b1c8a9f5473a3a9bfbf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:05.527574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9c36395091a4b1c8a9f5473a3a9bfbf'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:05.537722 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1234/2509] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:05 2022] DELETE /identity/v3/users/d9c36395091a4b1c8a9f5473a3a9bfbf => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:05.539118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d167c74-8b2c-4dec-bd32-9b9aab261c98 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:05.539663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d167c74-8b2c-4dec-bd32-9b9aab261c98 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:05.539663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d167c74-8b2c-4dec-bd32-9b9aab261c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:05.565599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5d167c74-8b2c-4dec-bd32-9b9aab261c98 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:05.567068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5d167c74-8b2c-4dec-bd32-9b9aab261c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:05.640219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1276/2510] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:41:05.651471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:05.658922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:05.673286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:05.720324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df050d5-fb81-4b93-adb2-bcc99e3a9b03 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:05.723991 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f9c73-9719-46f6-9961-d695c3426c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:05.725126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f9c73-9719-46f6-9961-d695c3426c52 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:05.725403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f9c73-9719-46f6-9961-d695c3426c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:05.725812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f9c73-9719-46f6-9961-d695c3426c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:05.738998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f9c73-9719-46f6-9961-d695c3426c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:05.741197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f9c73-9719-46f6-9961-d695c3426c52 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:05.752289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1235/2511] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:06.187054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d167c74-8b2c-4dec-bd32-9b9aab261c98 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:06.188609 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5d167c74-8b2c-4dec-bd32-9b9aab261c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:06.200980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d167c74-8b2c-4dec-bd32-9b9aab261c98 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:06.205740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:06.206376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:06.206655 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:06.206929 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] PATH_INFO: `/v3/projects/7cc3640de35b4364bc3ad8475b10d1e7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:06.208056 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc3640de35b4364bc3ad8475b10d1e7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:06.210512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:06.345092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc3640de35b4364bc3ad8475b10d1e7', 'request_id': 'req-ff807f4c-6e6a-412d-8022-e29655e858c5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:06.412923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc3640de35b4364bc3ad8475b10d1e7', 'request_id': 'req-ff807f4c-6e6a-412d-8022-e29655e858c5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:06.419394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc3640de35b4364bc3ad8475b10d1e7', 'request_id': 'req-ff807f4c-6e6a-412d-8022-e29655e858c5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:06.430052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] The token cache is being invalidate because project 7cc3640de35b4364bc3ad8475b10d1e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:06.430339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:06.430960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:06.432440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1277/2512] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:06 2022] DELETE /identity/v3/projects/7cc3640de35b4364bc3ad8475b10d1e7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:06.436629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f9c73-9719-46f6-9961-d695c3426c52 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:06.447037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f9c73-9719-46f6-9961-d695c3426c52 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:06.501570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:06.502192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:06.502468 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:06.502740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] PATH_INFO: `/v3/users/8760c253066d4a1d9c36f11eaa8db419` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:06.503837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8760c253066d4a1d9c36f11eaa8db419)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:06.506283 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:06.661532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8760c253066d4a1d9c36f11eaa8db419'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:06.662253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8760c253066d4a1d9c36f11eaa8db419'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:06.738047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8760c253066d4a1d9c36f11eaa8db419'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:06.752666 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8760c253066d4a1d9c36f11eaa8db419'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:06.760238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7df81d59-058e-4840-9d26-bcd4dc4c3184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8760c253066d4a1d9c36f11eaa8db419'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:06.766054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1236/2513] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:06 2022] DELETE /identity/v3/users/8760c253066d4a1d9c36f11eaa8db419 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:07.161674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:07.163245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:07.175942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff807f4c-6e6a-412d-8022-e29655e858c5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:07.207024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:07.239022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:07.240176 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:07.240459 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:07.240735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] PATH_INFO: `/v3/projects/423a1a8972874e96a49c934be2c93c55` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:07.242128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=423a1a8972874e96a49c934be2c93c55)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:07.244222 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:07.377179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423a1a8972874e96a49c934be2c93c55', 'request_id': 'req-e01662c1-c840-491b-8190-8f3862576edc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:07.481428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423a1a8972874e96a49c934be2c93c55', 'request_id': 'req-e01662c1-c840-491b-8190-8f3862576edc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:07.487963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423a1a8972874e96a49c934be2c93c55', 'request_id': 'req-e01662c1-c840-491b-8190-8f3862576edc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:07.499287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] The token cache is being invalidate because project 423a1a8972874e96a49c934be2c93c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:07.499575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:07.500200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e01662c1-c840-491b-8190-8f3862576edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:07.502197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1278/2514] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:07 2022] DELETE /identity/v3/projects/423a1a8972874e96a49c934be2c93c55 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:07.507496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55518969-5158-407c-9e5f-30a25183a12b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:07.507976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55518969-5158-407c-9e5f-30a25183a12b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:07.508082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55518969-5158-407c-9e5f-30a25183a12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:07.527454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-55518969-5158-407c-9e5f-30a25183a12b None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:07.528633 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-55518969-5158-407c-9e5f-30a25183a12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:07.559919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:07.588627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1237/2515] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:07.596333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2477a-8e48-4b15-a2dc-5307f32cf96f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:07.599753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2477a-8e48-4b15-a2dc-5307f32cf96f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:07.599753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2477a-8e48-4b15-a2dc-5307f32cf96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:07.616346 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dee2477a-8e48-4b15-a2dc-5307f32cf96f None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:07.618227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dee2477a-8e48-4b15-a2dc-5307f32cf96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:07.626134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1279/2516] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:07.630349 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55518969-5158-407c-9e5f-30a25183a12b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:07.641061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55518969-5158-407c-9e5f-30a25183a12b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:07.645644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a25d5-8901-4f08-9382-7331cede73f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:07.646529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a25d5-8901-4f08-9382-7331cede73f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:07.646992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a25d5-8901-4f08-9382-7331cede73f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:07.647626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a25d5-8901-4f08-9382-7331cede73f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:07.648709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a25d5-8901-4f08-9382-7331cede73f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:41:07.649075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a25d5-8901-4f08-9382-7331cede73f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:07.650736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a25d5-8901-4f08-9382-7331cede73f4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:07.662510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1238/2517] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:41:07.667020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2477a-8e48-4b15-a2dc-5307f32cf96f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:07.676265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2477a-8e48-4b15-a2dc-5307f32cf96f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:07.681584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477f7b5-78ad-4a4a-8b6c-739a4b00c088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:07.682179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477f7b5-78ad-4a4a-8b6c-739a4b00c088 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:07.682465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477f7b5-78ad-4a4a-8b6c-739a4b00c088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:07.682735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477f7b5-78ad-4a4a-8b6c-739a4b00c088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:07.683356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477f7b5-78ad-4a4a-8b6c-739a4b00c088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:07.685242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477f7b5-78ad-4a4a-8b6c-739a4b00c088 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:07.768609 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1280/2518] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:41:07 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:07.772841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a25d5-8901-4f08-9382-7331cede73f4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:07.782873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a25d5-8901-4f08-9382-7331cede73f4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:07.839384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250e03c-2e0c-467e-aac4-6d1a55f58151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:07.839971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250e03c-2e0c-467e-aac4-6d1a55f58151 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:07.840243 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250e03c-2e0c-467e-aac4-6d1a55f58151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:07.840509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250e03c-2e0c-467e-aac4-6d1a55f58151 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:07.841128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250e03c-2e0c-467e-aac4-6d1a55f58151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:07.843057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250e03c-2e0c-467e-aac4-6d1a55f58151 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:07.921258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1239/2519] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:41:07 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:07.925557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477f7b5-78ad-4a4a-8b6c-739a4b00c088 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:07.938913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477f7b5-78ad-4a4a-8b6c-739a4b00c088 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:07.943749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e4b63-036f-444e-9427-5096e749a53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:07.944341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e4b63-036f-444e-9427-5096e749a53a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:07.944664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e4b63-036f-444e-9427-5096e749a53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:07.945142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e4b63-036f-444e-9427-5096e749a53a admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:07.945907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e4b63-036f-444e-9427-5096e749a53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:07.947488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e4b63-036f-444e-9427-5096e749a53a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:07.957529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1281/2520] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:07.962178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250e03c-2e0c-467e-aac4-6d1a55f58151 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:07.972295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250e03c-2e0c-467e-aac4-6d1a55f58151 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:07.977412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce8128-8066-4fe1-8b22-43806983a98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:07.978028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce8128-8066-4fe1-8b22-43806983a98b admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:07.978308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce8128-8066-4fe1-8b22-43806983a98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:07.978576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce8128-8066-4fe1-8b22-43806983a98b admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:07.979207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce8128-8066-4fe1-8b22-43806983a98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:07.980795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce8128-8066-4fe1-8b22-43806983a98b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:07.992230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1240/2521] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:07.996511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e4b63-036f-444e-9427-5096e749a53a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:08.006538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e4b63-036f-444e-9427-5096e749a53a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:08.011297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa408cf-a136-4df8-890c-914dda52cb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:08.011945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa408cf-a136-4df8-890c-914dda52cb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:08.012225 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa408cf-a136-4df8-890c-914dda52cb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:08.012501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa408cf-a136-4df8-890c-914dda52cb60 admin admin] PATH_INFO: `/v3/projects/56b63c409c6c4109acd03b48276acab4/users/f7a22e9bf4654f3190d502afc5c3321d/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:08.026740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa408cf-a136-4df8-890c-914dda52cb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b63c409c6c4109acd03b48276acab4, user_id=f7a22e9bf4654f3190d502afc5c3321d, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:08.029320 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa408cf-a136-4df8-890c-914dda52cb60 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:08.086857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1282/2522] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:07 2022] PUT /identity/v3/projects/56b63c409c6c4109acd03b48276acab4/users/f7a22e9bf4654f3190d502afc5c3321d/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:41:08.098280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511063b3-d8da-4d03-8523-24ebda731da6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:08.098569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511063b3-d8da-4d03-8523-24ebda731da6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:08.098839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511063b3-d8da-4d03-8523-24ebda731da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:08.129132 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-511063b3-d8da-4d03-8523-24ebda731da6 None None] MFA Rules not processed for user `f7a22e9bf4654f3190d502afc5c3321d`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:08.130456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-511063b3-d8da-4d03-8523-24ebda731da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:08.196892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1241/2523] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:08.201587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa408cf-a136-4df8-890c-914dda52cb60 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:08.263921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:08.269129 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa408cf-a136-4df8-890c-914dda52cb60 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:08.274179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f786a0-6fdc-477f-8344-aade2995b92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:08.274791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f786a0-6fdc-477f-8344-aade2995b92f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:08.275075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f786a0-6fdc-477f-8344-aade2995b92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:08.275350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f786a0-6fdc-477f-8344-aade2995b92f admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:08.275988 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f786a0-6fdc-477f-8344-aade2995b92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:08.277942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f786a0-6fdc-477f-8344-aade2995b92f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:08.372747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1283/2524] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:41:08 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:08.373811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511063b3-d8da-4d03-8523-24ebda731da6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:08.385639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511063b3-d8da-4d03-8523-24ebda731da6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:08.418589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:08.448015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d9179-0f77-4ae0-960c-53e7716a1ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:08.448629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d9179-0f77-4ae0-960c-53e7716a1ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:08.448908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d9179-0f77-4ae0-960c-53e7716a1ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:08.449183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d9179-0f77-4ae0-960c-53e7716a1ce8 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:08.449853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d9179-0f77-4ae0-960c-53e7716a1ce8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:08.451798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d9179-0f77-4ae0-960c-53e7716a1ce8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:08.578388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1242/2525] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:41:08 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:08.582308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f786a0-6fdc-477f-8344-aade2995b92f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:08.592514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f786a0-6fdc-477f-8344-aade2995b92f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:08.597897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320332e-b14a-47ec-b524-7217c9ee32dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:08.598488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320332e-b14a-47ec-b524-7217c9ee32dd admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:08.598763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320332e-b14a-47ec-b524-7217c9ee32dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:08.599030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320332e-b14a-47ec-b524-7217c9ee32dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:08.599658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320332e-b14a-47ec-b524-7217c9ee32dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:08.601976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320332e-b14a-47ec-b524-7217c9ee32dd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:08.612232 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1284/2526] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:08.617094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d9179-0f77-4ae0-960c-53e7716a1ce8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:08.627136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d9179-0f77-4ae0-960c-53e7716a1ce8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:08.631794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed529c3d-7454-456f-9aa4-61d791ccf998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:08.632783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed529c3d-7454-456f-9aa4-61d791ccf998 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:08.633062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed529c3d-7454-456f-9aa4-61d791ccf998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:08.633332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed529c3d-7454-456f-9aa4-61d791ccf998 admin admin] PATH_INFO: `/v3/projects/ca098429ae154b69a7148452e46c65d1/users/dc23a42e3b3f48f8811b19cc97c21ab7/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:08.647706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed529c3d-7454-456f-9aa4-61d791ccf998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca098429ae154b69a7148452e46c65d1, user_id=dc23a42e3b3f48f8811b19cc97c21ab7, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:08.650765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed529c3d-7454-456f-9aa4-61d791ccf998 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:08.703971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1243/2527] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:08 2022] PUT /identity/v3/projects/ca098429ae154b69a7148452e46c65d1/users/dc23a42e3b3f48f8811b19cc97c21ab7/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:08.707971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320332e-b14a-47ec-b524-7217c9ee32dd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:08.719276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320332e-b14a-47ec-b524-7217c9ee32dd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:08.776877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5799bf-6a7c-486a-8e82-8b042eac4d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:08.777513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5799bf-6a7c-486a-8e82-8b042eac4d52 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:08.777801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5799bf-6a7c-486a-8e82-8b042eac4d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:08.778075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5799bf-6a7c-486a-8e82-8b042eac4d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:08.778712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5799bf-6a7c-486a-8e82-8b042eac4d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:08.780319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5799bf-6a7c-486a-8e82-8b042eac4d52 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:08.790562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1285/2528] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:41:08.794767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed529c3d-7454-456f-9aa4-61d791ccf998 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:08.805508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed529c3d-7454-456f-9aa4-61d791ccf998 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:08.810040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d579a-53e5-4392-a157-ddedc3291e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:08.810676 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d579a-53e5-4392-a157-ddedc3291e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:08.811001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d579a-53e5-4392-a157-ddedc3291e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:08.811214 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d579a-53e5-4392-a157-ddedc3291e88 admin admin] PATH_INFO: `/v3/projects/ca098429ae154b69a7148452e46c65d1/users/dc23a42e3b3f48f8811b19cc97c21ab7/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:08.813121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d579a-53e5-4392-a157-ddedc3291e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca098429ae154b69a7148452e46c65d1, user_id=dc23a42e3b3f48f8811b19cc97c21ab7, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:08.815778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d579a-53e5-4392-a157-ddedc3291e88 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:08.826348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1244/2529] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:08 2022] PUT /identity/v3/projects/ca098429ae154b69a7148452e46c65d1/users/dc23a42e3b3f48f8811b19cc97c21ab7/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:08.836617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933b1b9-6a43-47dd-8086-347d1aa3e54a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:08.836909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933b1b9-6a43-47dd-8086-347d1aa3e54a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:08.837187 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933b1b9-6a43-47dd-8086-347d1aa3e54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:08.865684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2933b1b9-6a43-47dd-8086-347d1aa3e54a None None] MFA Rules not processed for user `dc23a42e3b3f48f8811b19cc97c21ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:08.867713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2933b1b9-6a43-47dd-8086-347d1aa3e54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:08.935783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1286/2530] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:08.948496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d579a-53e5-4392-a157-ddedc3291e88 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:08.950210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ea8d579a-53e5-4392-a157-ddedc3291e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:09.009166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d579a-53e5-4392-a157-ddedc3291e88 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:09.012879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0f979-1ff5-4029-be35-f86df19c7a22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:09.013513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0f979-1ff5-4029-be35-f86df19c7a22 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:09.013799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0f979-1ff5-4029-be35-f86df19c7a22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:09.014072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0f979-1ff5-4029-be35-f86df19c7a22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:09.023157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0f979-1ff5-4029-be35-f86df19c7a22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:09.024592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0f979-1ff5-4029-be35-f86df19c7a22 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:09.033998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1245/2531] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:41:09.186961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933b1b9-6a43-47dd-8086-347d1aa3e54a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:09.197607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933b1b9-6a43-47dd-8086-347d1aa3e54a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:09.201237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76974a8-26b9-4ab5-ae58-e4aa3c1bfc9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:09.201877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76974a8-26b9-4ab5-ae58-e4aa3c1bfc9f service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:09.202421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76974a8-26b9-4ab5-ae58-e4aa3c1bfc9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:09.202421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76974a8-26b9-4ab5-ae58-e4aa3c1bfc9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:09.248726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:09.258807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76974a8-26b9-4ab5-ae58-e4aa3c1bfc9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:09.260246 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76974a8-26b9-4ab5-ae58-e4aa3c1bfc9f service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:09.269406 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1287/2532] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:09.776607 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0f979-1ff5-4029-be35-f86df19c7a22 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:09.839603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0f979-1ff5-4029-be35-f86df19c7a22 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:09.844045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:09.844656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:09.844929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:09.845197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] PATH_INFO: `/v3/users/f7a22e9bf4654f3190d502afc5c3321d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:09.846324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7a22e9bf4654f3190d502afc5c3321d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:09.848227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:09.935393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:10.055454 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7a22e9bf4654f3190d502afc5c3321d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:10.056382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7a22e9bf4654f3190d502afc5c3321d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:10.127229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7a22e9bf4654f3190d502afc5c3321d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:10.140974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7a22e9bf4654f3190d502afc5c3321d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:10.149075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7a22e9bf4654f3190d502afc5c3321d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:10.156000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1246/2533] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:09 2022] DELETE /identity/v3/users/f7a22e9bf4654f3190d502afc5c3321d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:10.160268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835332c-3b28-42fb-8d83-b67b327ffdaa None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:10.160624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835332c-3b28-42fb-8d83-b67b327ffdaa None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:10.160992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835332c-3b28-42fb-8d83-b67b327ffdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:10.180855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d835332c-3b28-42fb-8d83-b67b327ffdaa None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:10.182189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d835332c-3b28-42fb-8d83-b67b327ffdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:10.242050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1288/2534] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:10.252013 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:10.253412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:10.323880 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc4113-b4ed-4e9f-92a8-0842bd80a2ff admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:10.327474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58343d2-239f-4177-9888-4d4920803035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:10.328121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58343d2-239f-4177-9888-4d4920803035 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:10.328371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58343d2-239f-4177-9888-4d4920803035 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:10.328645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58343d2-239f-4177-9888-4d4920803035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:10.338887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58343d2-239f-4177-9888-4d4920803035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:10.340403 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58343d2-239f-4177-9888-4d4920803035 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:10.350360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1247/2535] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:10.814372 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835332c-3b28-42fb-8d83-b67b327ffdaa None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:10.815917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d835332c-3b28-42fb-8d83-b67b327ffdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:10.831670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835332c-3b28-42fb-8d83-b67b327ffdaa None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:10.837817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:10.838457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:10.838734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:10.839002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] PATH_INFO: `/v3/projects/56b63c409c6c4109acd03b48276acab4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:10.840216 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b63c409c6c4109acd03b48276acab4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:10.842120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:10.998024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b63c409c6c4109acd03b48276acab4', 'request_id': 'req-06110906-2bcc-4c64-9b95-a6400fb2ea33'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:11.104281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b63c409c6c4109acd03b48276acab4', 'request_id': 'req-06110906-2bcc-4c64-9b95-a6400fb2ea33'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:11.111118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b63c409c6c4109acd03b48276acab4', 'request_id': 'req-06110906-2bcc-4c64-9b95-a6400fb2ea33'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:11.122684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] The token cache is being invalidate because project 56b63c409c6c4109acd03b48276acab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:11.122945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:11.123603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:11.125097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1289/2536] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:10 2022] DELETE /identity/v3/projects/56b63c409c6c4109acd03b48276acab4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:11.129163 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58343d2-239f-4177-9888-4d4920803035 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:11.138847 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58343d2-239f-4177-9888-4d4920803035 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:11.196348 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:11.196955 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:11.197227 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:11.197549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] PATH_INFO: `/v3/users/dc23a42e3b3f48f8811b19cc97c21ab7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:11.198639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc23a42e3b3f48f8811b19cc97c21ab7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:11.200706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:11.386229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc23a42e3b3f48f8811b19cc97c21ab7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:11.386836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc23a42e3b3f48f8811b19cc97c21ab7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:11.389347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:11.464305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc23a42e3b3f48f8811b19cc97c21ab7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:11.477040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc23a42e3b3f48f8811b19cc97c21ab7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:11.483731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-983f2415-646e-41b3-b4d8-4bddf955be79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc23a42e3b3f48f8811b19cc97c21ab7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:11.489784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1248/2537] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:11 2022] DELETE /identity/v3/users/dc23a42e3b3f48f8811b19cc97c21ab7 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:11.894327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:11.895856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:11.910212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06110906-2bcc-4c64-9b95-a6400fb2ea33 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:11.979440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:11.985284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:11.985284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:11.985284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] PATH_INFO: `/v3/projects/ca098429ae154b69a7148452e46c65d1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:11.985284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca098429ae154b69a7148452e46c65d1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:11.985284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:12.005754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:12.064796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca098429ae154b69a7148452e46c65d1', 'request_id': 'req-a0611ccf-4018-4486-9dbd-ea0df5637680'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:12.138234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca098429ae154b69a7148452e46c65d1', 'request_id': 'req-a0611ccf-4018-4486-9dbd-ea0df5637680'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:12.144915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca098429ae154b69a7148452e46c65d1', 'request_id': 'req-a0611ccf-4018-4486-9dbd-ea0df5637680'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:12.156633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] The token cache is being invalidate because project ca098429ae154b69a7148452e46c65d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:12.157708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:12.157708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0611ccf-4018-4486-9dbd-ea0df5637680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:12.159249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1290/2538] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:11 2022] DELETE /identity/v3/projects/ca098429ae154b69a7148452e46c65d1 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:12.164343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a0fd5-7596-4334-aeb2-9159f773ede9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:12.165504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a0fd5-7596-4334-aeb2-9159f773ede9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:12.165953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a0fd5-7596-4334-aeb2-9159f773ede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:12.185409 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-014a0fd5-7596-4334-aeb2-9159f773ede9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:12.186626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-014a0fd5-7596-4334-aeb2-9159f773ede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:12.246262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1249/2539] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:12.254963 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225d8b-91ce-495f-a968-7c51b88e8fdf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:12.255248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225d8b-91ce-495f-a968-7c51b88e8fdf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:12.255521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225d8b-91ce-495f-a968-7c51b88e8fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:12.275660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-42225d8b-91ce-495f-a968-7c51b88e8fdf None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:12.276844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-42225d8b-91ce-495f-a968-7c51b88e8fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:12.284575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1291/2540] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:12.288714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a0fd5-7596-4334-aeb2-9159f773ede9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:12.299104 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a0fd5-7596-4334-aeb2-9159f773ede9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:12.303575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec618c28-8d5e-405c-84ab-fd2ae0633175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:12.304155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec618c28-8d5e-405c-84ab-fd2ae0633175 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:12.304428 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec618c28-8d5e-405c-84ab-fd2ae0633175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:12.304695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec618c28-8d5e-405c-84ab-fd2ae0633175 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:12.305616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec618c28-8d5e-405c-84ab-fd2ae0633175 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:41:12.305964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec618c28-8d5e-405c-84ab-fd2ae0633175 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:12.307529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec618c28-8d5e-405c-84ab-fd2ae0633175 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:12.319067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1250/2541] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:41:12.323691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42225d8b-91ce-495f-a968-7c51b88e8fdf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:12.332670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42225d8b-91ce-495f-a968-7c51b88e8fdf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:12.340207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376df2e-27f6-4141-854c-9a46d63fdd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:12.340795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376df2e-27f6-4141-854c-9a46d63fdd99 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:12.341069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376df2e-27f6-4141-854c-9a46d63fdd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:12.341333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376df2e-27f6-4141-854c-9a46d63fdd99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:12.342001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376df2e-27f6-4141-854c-9a46d63fdd99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:12.343893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376df2e-27f6-4141-854c-9a46d63fdd99 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:12.436747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1292/2542] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:41:12 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:12.440812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec618c28-8d5e-405c-84ab-fd2ae0633175 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:12.451766 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec618c28-8d5e-405c-84ab-fd2ae0633175 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:12.507540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff2776-4b64-41c1-964c-9a32b46458df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:12.508128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2776-4b64-41c1-964c-9a32b46458df admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:12.508966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2776-4b64-41c1-964c-9a32b46458df admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:12.509286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2776-4b64-41c1-964c-9a32b46458df admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:12.510000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff2776-4b64-41c1-964c-9a32b46458df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:12.511909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff2776-4b64-41c1-964c-9a32b46458df admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:12.573266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1251/2543] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:41:12 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:12.577416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376df2e-27f6-4141-854c-9a46d63fdd99 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:12.588796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376df2e-27f6-4141-854c-9a46d63fdd99 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:12.593708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb97ad-2c44-48f5-adba-93f465c9e6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:12.594362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb97ad-2c44-48f5-adba-93f465c9e6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:12.594642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb97ad-2c44-48f5-adba-93f465c9e6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:12.594913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb97ad-2c44-48f5-adba-93f465c9e6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:12.595553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb97ad-2c44-48f5-adba-93f465c9e6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:12.597182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb97ad-2c44-48f5-adba-93f465c9e6c0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:12.607635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1293/2544] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:12.611932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff2776-4b64-41c1-964c-9a32b46458df admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:12.622250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff2776-4b64-41c1-964c-9a32b46458df admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:12.627073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8159f-677b-4661-8b24-c4c8883583b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:12.627761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8159f-677b-4661-8b24-c4c8883583b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:12.628042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8159f-677b-4661-8b24-c4c8883583b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:12.628381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8159f-677b-4661-8b24-c4c8883583b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:12.629046 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8159f-677b-4661-8b24-c4c8883583b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:12.630664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8159f-677b-4661-8b24-c4c8883583b8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:12.642156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1252/2545] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:12.647539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb97ad-2c44-48f5-adba-93f465c9e6c0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:12.657652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb97ad-2c44-48f5-adba-93f465c9e6c0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:12.662311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d5cba-0598-49f3-84bc-32a158da921b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:12.662966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5cba-0598-49f3-84bc-32a158da921b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:12.663242 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5cba-0598-49f3-84bc-32a158da921b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:12.663513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5cba-0598-49f3-84bc-32a158da921b admin admin] PATH_INFO: `/v3/projects/f31bf2972a0e47e6a1b1a2b45f6e5bd5/users/ae0e5328dd3f4ef2af31b1136481597a/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:12.679152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d5cba-0598-49f3-84bc-32a158da921b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f31bf2972a0e47e6a1b1a2b45f6e5bd5, user_id=ae0e5328dd3f4ef2af31b1136481597a, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:12.682519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d5cba-0598-49f3-84bc-32a158da921b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:12.750908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1294/2546] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:12 2022] PUT /identity/v3/projects/f31bf2972a0e47e6a1b1a2b45f6e5bd5/users/ae0e5328dd3f4ef2af31b1136481597a/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:12.766160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcc1e9-14d7-4005-8ad7-b8b52e2e818e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:12.766440 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcc1e9-14d7-4005-8ad7-b8b52e2e818e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:12.766912 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcc1e9-14d7-4005-8ad7-b8b52e2e818e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:12.794791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-20bcc1e9-14d7-4005-8ad7-b8b52e2e818e None None] MFA Rules not processed for user `ae0e5328dd3f4ef2af31b1136481597a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:12.795985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-20bcc1e9-14d7-4005-8ad7-b8b52e2e818e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:12.817373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:12.861734 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1253/2547] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:41:12.873295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d5cba-0598-49f3-84bc-32a158da921b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:12.875047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-472d5cba-0598-49f3-84bc-32a158da921b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:12.932207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d5cba-0598-49f3-84bc-32a158da921b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:12.935904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0988846-7770-4996-a30d-3781d99717b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c23597e0da2454abd2a5354d779b092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:12.936510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988846-7770-4996-a30d-3781d99717b3 service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:12.936788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988846-7770-4996-a30d-3781d99717b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:12.937062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988846-7770-4996-a30d-3781d99717b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:12.946539 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0988846-7770-4996-a30d-3781d99717b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:12.947976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0988846-7770-4996-a30d-3781d99717b3 service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:12.956815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1295/2548] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:22.952849 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:22.961200 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=17 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:23.653626 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba9f5-754f-424a-8700-9f917250db56 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:23.653919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba9f5-754f-424a-8700-9f917250db56 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:23.654192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba9f5-754f-424a-8700-9f917250db56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:23.706242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a8dba9f5-754f-424a-8700-9f917250db56 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:23.707455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a8dba9f5-754f-424a-8700-9f917250db56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:23.780706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1254/2549] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 1 05:41:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:23.794167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0988846-7770-4996-a30d-3781d99717b3 service glance] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:23.795436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e0988846-7770-4996-a30d-3781d99717b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:23.889376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0988846-7770-4996-a30d-3781d99717b3 service glance] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:23.894507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352137a-445e-4b10-85c5-6766a5fe0b1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:23.895123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352137a-445e-4b10-85c5-6766a5fe0b1c service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:23.895402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352137a-445e-4b10-85c5-6766a5fe0b1c service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:23.895672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352137a-445e-4b10-85c5-6766a5fe0b1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:23.905033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352137a-445e-4b10-85c5-6766a5fe0b1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:23.906648 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352137a-445e-4b10-85c5-6766a5fe0b1c service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:23.914646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1296/2550] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:41:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:41:31.806555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba9f5-754f-424a-8700-9f917250db56 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:31.808315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a8dba9f5-754f-424a-8700-9f917250db56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:31.864775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba9f5-754f-424a-8700-9f917250db56 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:31.868336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35bf78-3a9d-47bf-bb5b-c84984f41b3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:31.868926 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35bf78-3a9d-47bf-bb5b-c84984f41b3b service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:31.869218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35bf78-3a9d-47bf-bb5b-c84984f41b3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:31.869755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35bf78-3a9d-47bf-bb5b-c84984f41b3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:31.871629 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ac35bf78-3a9d-47bf-bb5b-c84984f41b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:31.935756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35bf78-3a9d-47bf-bb5b-c84984f41b3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:31.937207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35bf78-3a9d-47bf-bb5b-c84984f41b3b service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:31.947188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1255/2551] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:33.911534 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=41 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:35.639416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352137a-445e-4b10-85c5-6766a5fe0b1c service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:35.641479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e352137a-445e-4b10-85c5-6766a5fe0b1c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:35.658473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352137a-445e-4b10-85c5-6766a5fe0b1c service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:35.663097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:35.663780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:35.664169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:35.664447 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] PATH_INFO: `/v3/users/b9db0f3de10842eb97dc6fc580f4489a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:35.678937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9db0f3de10842eb97dc6fc580f4489a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:35.680899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:35.820988 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9db0f3de10842eb97dc6fc580f4489a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:35.834094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9db0f3de10842eb97dc6fc580f4489a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:35.842024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9db0f3de10842eb97dc6fc580f4489a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:35.905048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9db0f3de10842eb97dc6fc580f4489a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:35.905727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f0cb5d1-956b-420e-9af6-e1da450e8cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9db0f3de10842eb97dc6fc580f4489a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:35.916050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1297/2552] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:35 2022] DELETE /identity/v3/users/b9db0f3de10842eb97dc6fc580f4489a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:36.154958 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35bf78-3a9d-47bf-bb5b-c84984f41b3b service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:36.165056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35bf78-3a9d-47bf-bb5b-c84984f41b3b service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:36.188802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:36.230401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:36.231067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:36.231345 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:36.231613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] PATH_INFO: `/v3/projects/d02b8bfcfa134334858ed8b7981348f1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:36.242496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d02b8bfcfa134334858ed8b7981348f1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:36.245277 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:36.356189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd02b8bfcfa134334858ed8b7981348f1', 'request_id': 'req-cfd5120b-0dda-4333-99b6-6de7ccb39b84'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:36.441140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd02b8bfcfa134334858ed8b7981348f1', 'request_id': 'req-cfd5120b-0dda-4333-99b6-6de7ccb39b84'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:36.448417 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd02b8bfcfa134334858ed8b7981348f1', 'request_id': 'req-cfd5120b-0dda-4333-99b6-6de7ccb39b84'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:36.459160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] The token cache is being invalidate because project d02b8bfcfa134334858ed8b7981348f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:36.459447 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:36.460077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cfd5120b-0dda-4333-99b6-6de7ccb39b84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:36.461569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1256/2553] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:36 2022] DELETE /identity/v3/projects/d02b8bfcfa134334858ed8b7981348f1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:36.466798 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eeb78-b0af-4f24-85d7-b52f1875aaf7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:36.467090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eeb78-b0af-4f24-85d7-b52f1875aaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:36.467368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eeb78-b0af-4f24-85d7-b52f1875aaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:36.487378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-730eeb78-b0af-4f24-85d7-b52f1875aaf7 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:36.488591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-730eeb78-b0af-4f24-85d7-b52f1875aaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:36.548201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1298/2554] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:36.555746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9c8b4-5210-4688-b2e3-1abe11059481 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:36.556050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9c8b4-5210-4688-b2e3-1abe11059481 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:36.556317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9c8b4-5210-4688-b2e3-1abe11059481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:36.576917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-31b9c8b4-5210-4688-b2e3-1abe11059481 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:36.578142 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-31b9c8b4-5210-4688-b2e3-1abe11059481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:36.585805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1257/2555] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:36.590312 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730eeb78-b0af-4f24-85d7-b52f1875aaf7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:36.600055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730eeb78-b0af-4f24-85d7-b52f1875aaf7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:36.605164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86032045-37c7-44dd-a32a-e3a54438cb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:36.605797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86032045-37c7-44dd-a32a-e3a54438cb8e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:36.606077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86032045-37c7-44dd-a32a-e3a54438cb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:36.606350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86032045-37c7-44dd-a32a-e3a54438cb8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:36.606996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86032045-37c7-44dd-a32a-e3a54438cb8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:41:36.607315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86032045-37c7-44dd-a32a-e3a54438cb8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:36.608882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86032045-37c7-44dd-a32a-e3a54438cb8e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:36.620906 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1299/2556] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:41:36.625207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9c8b4-5210-4688-b2e3-1abe11059481 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:36.634276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9c8b4-5210-4688-b2e3-1abe11059481 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:36.638969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d50cf9-a6ed-4ccc-8174-b0a0e5f77f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:36.639548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d50cf9-a6ed-4ccc-8174-b0a0e5f77f88 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:36.639822 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d50cf9-a6ed-4ccc-8174-b0a0e5f77f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:36.640090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d50cf9-a6ed-4ccc-8174-b0a0e5f77f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:36.640709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d50cf9-a6ed-4ccc-8174-b0a0e5f77f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:36.642598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d50cf9-a6ed-4ccc-8174-b0a0e5f77f88 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:36.717192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1258/2557] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:41:36 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:36.721243 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86032045-37c7-44dd-a32a-e3a54438cb8e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:36.732651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86032045-37c7-44dd-a32a-e3a54438cb8e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:36.776600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:36.788084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed99f6-9213-49d3-8b06-3665d50e4593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:36.789840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed99f6-9213-49d3-8b06-3665d50e4593 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:36.789840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed99f6-9213-49d3-8b06-3665d50e4593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:36.789840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed99f6-9213-49d3-8b06-3665d50e4593 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:36.789840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed99f6-9213-49d3-8b06-3665d50e4593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:36.792785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed99f6-9213-49d3-8b06-3665d50e4593 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:36.863874 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1300/2558] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:41:36 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:36.867935 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d50cf9-a6ed-4ccc-8174-b0a0e5f77f88 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:36.878249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d50cf9-a6ed-4ccc-8174-b0a0e5f77f88 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:36.882979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa6fc-0065-4099-9920-f62494c70e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:36.883572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa6fc-0065-4099-9920-f62494c70e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:36.883842 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa6fc-0065-4099-9920-f62494c70e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:36.884106 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa6fc-0065-4099-9920-f62494c70e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:36.884811 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fa6fc-0065-4099-9920-f62494c70e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:36.886564 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fa6fc-0065-4099-9920-f62494c70e4b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:36.896528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1259/2559] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:36.901042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed99f6-9213-49d3-8b06-3665d50e4593 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:36.910942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed99f6-9213-49d3-8b06-3665d50e4593 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:36.915505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac24fd-909b-4dd7-b437-c08853f47fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:36.916778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac24fd-909b-4dd7-b437-c08853f47fff admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:36.917245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac24fd-909b-4dd7-b437-c08853f47fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:36.917634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac24fd-909b-4dd7-b437-c08853f47fff admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:36.918272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac24fd-909b-4dd7-b437-c08853f47fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:36.919850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac24fd-909b-4dd7-b437-c08853f47fff admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:36.930448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1301/2560] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:36.934586 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa6fc-0065-4099-9920-f62494c70e4b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:36.944716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa6fc-0065-4099-9920-f62494c70e4b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:36.949741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedfa7c-af66-4851-8767-f0c38bd5aee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:36.950384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedfa7c-af66-4851-8767-f0c38bd5aee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:36.950657 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedfa7c-af66-4851-8767-f0c38bd5aee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:36.950924 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedfa7c-af66-4851-8767-f0c38bd5aee8 admin admin] PATH_INFO: `/v3/projects/8aaba82265b84828ac8acbd02a5a0941/users/b7f21cfedbd94762824620e3570d2d02/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:36.966671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedfa7c-af66-4851-8767-f0c38bd5aee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aaba82265b84828ac8acbd02a5a0941, user_id=b7f21cfedbd94762824620e3570d2d02, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:36.969421 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedfa7c-af66-4851-8767-f0c38bd5aee8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:37.031236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1260/2561] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:36 2022] PUT /identity/v3/projects/8aaba82265b84828ac8acbd02a5a0941/users/b7f21cfedbd94762824620e3570d2d02/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:37.042757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37761628-9d9f-44d3-bd09-ef01bf567c40 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:37.043040 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37761628-9d9f-44d3-bd09-ef01bf567c40 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:37.043313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37761628-9d9f-44d3-bd09-ef01bf567c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:37.071711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-37761628-9d9f-44d3-bd09-ef01bf567c40 None None] MFA Rules not processed for user `b7f21cfedbd94762824620e3570d2d02`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:37.073682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-37761628-9d9f-44d3-bd09-ef01bf567c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:37.139321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1302/2562] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:37.151509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedfa7c-af66-4851-8767-f0c38bd5aee8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:37.166652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3aedfa7c-af66-4851-8767-f0c38bd5aee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:37.200371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:37.231702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedfa7c-af66-4851-8767-f0c38bd5aee8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:37.237619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99244250-48e9-4a29-be6d-2cba816b1e2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:37.238222 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99244250-48e9-4a29-be6d-2cba816b1e2c service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:37.238499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99244250-48e9-4a29-be6d-2cba816b1e2c service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:37.238769 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99244250-48e9-4a29-be6d-2cba816b1e2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:37.248063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99244250-48e9-4a29-be6d-2cba816b1e2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:37.249675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99244250-48e9-4a29-be6d-2cba816b1e2c service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:37.258753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1261/2563] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:38.600643 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37761628-9d9f-44d3-bd09-ef01bf567c40 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:38.602067 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-37761628-9d9f-44d3-bd09-ef01bf567c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:38.667986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37761628-9d9f-44d3-bd09-ef01bf567c40 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:38.672486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:38.673114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:38.673391 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:38.673712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] PATH_INFO: `/v3/users/ae0e5328dd3f4ef2af31b1136481597a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:38.674831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0e5328dd3f4ef2af31b1136481597a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:38.676747 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:38.829742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0e5328dd3f4ef2af31b1136481597a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:38.837304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:38.843407 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0e5328dd3f4ef2af31b1136481597a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:38.850316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0e5328dd3f4ef2af31b1136481597a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:38.983993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0e5328dd3f4ef2af31b1136481597a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:38.984949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0e5328dd3f4ef2af31b1136481597a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:38.992005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1303/2564] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:38 2022] DELETE /identity/v3/users/ae0e5328dd3f4ef2af31b1136481597a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:41:38.997357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9dc2b-75ee-4e85-a1bf-c25c2eaee206 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:38.997867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9dc2b-75ee-4e85-a1bf-c25c2eaee206 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:38.997988 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9dc2b-75ee-4e85-a1bf-c25c2eaee206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:39.019347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1db9dc2b-75ee-4e85-a1bf-c25c2eaee206 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:39.020229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1db9dc2b-75ee-4e85-a1bf-c25c2eaee206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:39.080052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1262/2565] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:41:39.090073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:39.091918 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:39.152688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fcb12-437d-441a-ae0d-41b2e8a48dbd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:39.156809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dc17f-c6d7-4813-b4fc-68cdb5b7c1b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:39.157394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dc17f-c6d7-4813-b4fc-68cdb5b7c1b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:39.157780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dc17f-c6d7-4813-b4fc-68cdb5b7c1b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:39.158049 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dc17f-c6d7-4813-b4fc-68cdb5b7c1b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:39.168096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225dc17f-c6d7-4813-b4fc-68cdb5b7c1b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:39.169540 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225dc17f-c6d7-4813-b4fc-68cdb5b7c1b3 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:39.178953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1304/2566] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:39.575076 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9dc2b-75ee-4e85-a1bf-c25c2eaee206 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:39.576625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1db9dc2b-75ee-4e85-a1bf-c25c2eaee206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:39.589280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9dc2b-75ee-4e85-a1bf-c25c2eaee206 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:39.594382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:39.594986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:39.596121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:39.596121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] PATH_INFO: `/v3/projects/f31bf2972a0e47e6a1b1a2b45f6e5bd5` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:39.596614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f31bf2972a0e47e6a1b1a2b45f6e5bd5)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:39.598580 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:39.681168 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f31bf2972a0e47e6a1b1a2b45f6e5bd5', 'request_id': 'req-da2677b4-bdd2-4ca0-9f79-d2adceed4112'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:39.732667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f31bf2972a0e47e6a1b1a2b45f6e5bd5', 'request_id': 'req-da2677b4-bdd2-4ca0-9f79-d2adceed4112'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:39.739534 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f31bf2972a0e47e6a1b1a2b45f6e5bd5', 'request_id': 'req-da2677b4-bdd2-4ca0-9f79-d2adceed4112'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:39.750810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] The token cache is being invalidate because project f31bf2972a0e47e6a1b1a2b45f6e5bd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:39.751054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:39.751667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-da2677b4-bdd2-4ca0-9f79-d2adceed4112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:39.753137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1263/2567] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:39 2022] DELETE /identity/v3/projects/f31bf2972a0e47e6a1b1a2b45f6e5bd5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:39.758084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d4944-8936-45f5-8728-a0ca0459c1b5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:39.758560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d4944-8936-45f5-8728-a0ca0459c1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:39.758671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d4944-8936-45f5-8728-a0ca0459c1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:39.778467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8f9d4944-8936-45f5-8728-a0ca0459c1b5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:39.779664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8f9d4944-8936-45f5-8728-a0ca0459c1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:39.839708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1305/2568] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:39.847497 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c2166-cf66-47d5-a127-a1d202745fde None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:39.847764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c2166-cf66-47d5-a127-a1d202745fde None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:39.848033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c2166-cf66-47d5-a127-a1d202745fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:39.867544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-804c2166-cf66-47d5-a127-a1d202745fde None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:39.868728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-804c2166-cf66-47d5-a127-a1d202745fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:39.877221 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1264/2569] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:39.881524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d4944-8936-45f5-8728-a0ca0459c1b5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:39.891434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d4944-8936-45f5-8728-a0ca0459c1b5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:39.895864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd0f19-2c97-41c2-9b4e-d36ab6ea9b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:39.896448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd0f19-2c97-41c2-9b4e-d36ab6ea9b53 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:39.896725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd0f19-2c97-41c2-9b4e-d36ab6ea9b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:39.897006 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd0f19-2c97-41c2-9b4e-d36ab6ea9b53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:39.897661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd0f19-2c97-41c2-9b4e-d36ab6ea9b53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:41:39.897961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd0f19-2c97-41c2-9b4e-d36ab6ea9b53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:39.899556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd0f19-2c97-41c2-9b4e-d36ab6ea9b53 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:39.911642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1306/2570] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:41:39.915879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c2166-cf66-47d5-a127-a1d202745fde None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:39.926009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c2166-cf66-47d5-a127-a1d202745fde None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:39.930618 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caeffa-69ff-4313-9612-62ee2059a7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:39.931280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caeffa-69ff-4313-9612-62ee2059a7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:39.931485 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caeffa-69ff-4313-9612-62ee2059a7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:39.931983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caeffa-69ff-4313-9612-62ee2059a7f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:39.932606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caeffa-69ff-4313-9612-62ee2059a7f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:39.934545 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caeffa-69ff-4313-9612-62ee2059a7f1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:40.014344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1265/2571] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:41:39 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:40.018589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd0f19-2c97-41c2-9b4e-d36ab6ea9b53 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:40.029296 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd0f19-2c97-41c2-9b4e-d36ab6ea9b53 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:40.085446 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a63cac-8c5d-4953-a72f-29d949c655ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:40.086077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a63cac-8c5d-4953-a72f-29d949c655ed admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:40.086356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a63cac-8c5d-4953-a72f-29d949c655ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:40.086629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a63cac-8c5d-4953-a72f-29d949c655ed admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:40.087251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a63cac-8c5d-4953-a72f-29d949c655ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:40.089907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a63cac-8c5d-4953-a72f-29d949c655ed admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:40.100761 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:40.149836 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1307/2572] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:41:40 2022] POST /identity/v3/users => generated 282 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:41:40.157694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caeffa-69ff-4313-9612-62ee2059a7f1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:40.168274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caeffa-69ff-4313-9612-62ee2059a7f1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:40.172870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7a375-1a32-44d8-9332-56217fb59b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:40.173475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7a375-1a32-44d8-9332-56217fb59b71 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:40.177754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7a375-1a32-44d8-9332-56217fb59b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:40.177754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7a375-1a32-44d8-9332-56217fb59b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:40.177754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7a375-1a32-44d8-9332-56217fb59b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:40.177754 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7a375-1a32-44d8-9332-56217fb59b71 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:40.186723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1266/2573] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:40.190786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a63cac-8c5d-4953-a72f-29d949c655ed admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:40.201574 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a63cac-8c5d-4953-a72f-29d949c655ed admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:40.206377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea41d8e-02d5-46c2-9224-18420db15125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:40.206965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea41d8e-02d5-46c2-9224-18420db15125 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:40.207244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea41d8e-02d5-46c2-9224-18420db15125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:40.207515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea41d8e-02d5-46c2-9224-18420db15125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:40.208153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea41d8e-02d5-46c2-9224-18420db15125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:40.209757 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea41d8e-02d5-46c2-9224-18420db15125 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:40.220680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1308/2574] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:40.224939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7a375-1a32-44d8-9332-56217fb59b71 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:40.235652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7a375-1a32-44d8-9332-56217fb59b71 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:40.240377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a826c84-d22a-4ef5-86a1-8ad5e50578ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:40.241051 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a826c84-d22a-4ef5-86a1-8ad5e50578ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:40.241331 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a826c84-d22a-4ef5-86a1-8ad5e50578ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:40.241653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a826c84-d22a-4ef5-86a1-8ad5e50578ad admin admin] PATH_INFO: `/v3/projects/0d8c07e4bf2b4e779e8dac2e372ab6df/users/aae99e0f71874495bef6660a6419e676/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:40.245652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:40.256917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a826c84-d22a-4ef5-86a1-8ad5e50578ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8c07e4bf2b4e779e8dac2e372ab6df, user_id=aae99e0f71874495bef6660a6419e676, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:40.259591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a826c84-d22a-4ef5-86a1-8ad5e50578ad admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:40.361651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1267/2575] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:40 2022] PUT /identity/v3/projects/0d8c07e4bf2b4e779e8dac2e372ab6df/users/aae99e0f71874495bef6660a6419e676/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:40.373241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70653ab-5b15-4307-bb63-69ecb6577604 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:40.375624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70653ab-5b15-4307-bb63-69ecb6577604 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:40.375624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70653ab-5b15-4307-bb63-69ecb6577604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:40.402319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c70653ab-5b15-4307-bb63-69ecb6577604 None None] MFA Rules not processed for user `aae99e0f71874495bef6660a6419e676`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:40.404300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c70653ab-5b15-4307-bb63-69ecb6577604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:40.469279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1309/2576] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:40.481406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a826c84-d22a-4ef5-86a1-8ad5e50578ad admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:40.483113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6a826c84-d22a-4ef5-86a1-8ad5e50578ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:40.542315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a826c84-d22a-4ef5-86a1-8ad5e50578ad admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:40.545977 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee852b-7f6e-4ad7-a2be-7a862af38e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:40.546577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee852b-7f6e-4ad7-a2be-7a862af38e86 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:40.546854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee852b-7f6e-4ad7-a2be-7a862af38e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:40.547124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee852b-7f6e-4ad7-a2be-7a862af38e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:40.557316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee852b-7f6e-4ad7-a2be-7a862af38e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:40.558987 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee852b-7f6e-4ad7-a2be-7a862af38e86 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:40.567953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1268/2577] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:47.410079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70653ab-5b15-4307-bb63-69ecb6577604 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:47.410079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c70653ab-5b15-4307-bb63-69ecb6577604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:47.473511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70653ab-5b15-4307-bb63-69ecb6577604 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:47.477881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:47.478500 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:47.478776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:47.479047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] PATH_INFO: `/v3/users/b7f21cfedbd94762824620e3570d2d02` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:47.480897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7f21cfedbd94762824620e3570d2d02)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:47.482855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:47.632680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7f21cfedbd94762824620e3570d2d02'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:47.644480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7f21cfedbd94762824620e3570d2d02'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:47.651223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7f21cfedbd94762824620e3570d2d02'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:47.654033 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:47.738706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7f21cfedbd94762824620e3570d2d02'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:47.739321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7f21cfedbd94762824620e3570d2d02'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:47.746489 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1310/2578] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:47 2022] DELETE /identity/v3/users/b7f21cfedbd94762824620e3570d2d02 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:47.751040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210ecde-e705-4882-b0cd-20a7030b7b7c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:47.751499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210ecde-e705-4882-b0cd-20a7030b7b7c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:47.751605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210ecde-e705-4882-b0cd-20a7030b7b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:47.771681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f210ecde-e705-4882-b0cd-20a7030b7b7c None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:47.772343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f210ecde-e705-4882-b0cd-20a7030b7b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:47.837736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1269/2579] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:47.847821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:47.849268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:47.907706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efc083-e115-4025-90f2-602165d8762b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:47.911401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25047ed-7b34-4c39-b7d2-36dde275c60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:47.912005 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25047ed-7b34-4c39-b7d2-36dde275c60b service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:47.912286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25047ed-7b34-4c39-b7d2-36dde275c60b service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:47.912559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25047ed-7b34-4c39-b7d2-36dde275c60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:47.922475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25047ed-7b34-4c39-b7d2-36dde275c60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:47.923897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25047ed-7b34-4c39-b7d2-36dde275c60b service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:47.933560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1311/2580] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:48.346517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210ecde-e705-4882-b0cd-20a7030b7b7c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:48.347957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f210ecde-e705-4882-b0cd-20a7030b7b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:48.359969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210ecde-e705-4882-b0cd-20a7030b7b7c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:48.366905 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:48.367402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:48.367681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:48.367952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] PATH_INFO: `/v3/projects/8aaba82265b84828ac8acbd02a5a0941` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:48.369090 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aaba82265b84828ac8acbd02a5a0941)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:48.372100 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:48.476144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aaba82265b84828ac8acbd02a5a0941', 'request_id': 'req-447c0178-1913-4d52-886d-fff5c992ed4a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:48.538259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aaba82265b84828ac8acbd02a5a0941', 'request_id': 'req-447c0178-1913-4d52-886d-fff5c992ed4a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:48.545448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aaba82265b84828ac8acbd02a5a0941', 'request_id': 'req-447c0178-1913-4d52-886d-fff5c992ed4a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:48.556204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] The token cache is being invalidate because project 8aaba82265b84828ac8acbd02a5a0941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:48.556588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:48.557219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-447c0178-1913-4d52-886d-fff5c992ed4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:48.559619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1270/2581] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:48 2022] DELETE /identity/v3/projects/8aaba82265b84828ac8acbd02a5a0941 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:48.564476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7814d4-bdcd-4d86-8a64-8ef1c0f88763 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:48.564952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7814d4-bdcd-4d86-8a64-8ef1c0f88763 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:48.565050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7814d4-bdcd-4d86-8a64-8ef1c0f88763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:48.584759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ae7814d4-bdcd-4d86-8a64-8ef1c0f88763 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:48.586059 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ae7814d4-bdcd-4d86-8a64-8ef1c0f88763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:48.646010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1312/2582] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:41:48.654436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2da8d7-03bc-411f-ac60-a72e91ad8eb1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:48.654718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2da8d7-03bc-411f-ac60-a72e91ad8eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:48.654991 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2da8d7-03bc-411f-ac60-a72e91ad8eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:48.675448 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4b2da8d7-03bc-411f-ac60-a72e91ad8eb1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:48.676644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4b2da8d7-03bc-411f-ac60-a72e91ad8eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:48.684782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1271/2583] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:48.688932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7814d4-bdcd-4d86-8a64-8ef1c0f88763 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:48.698964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7814d4-bdcd-4d86-8a64-8ef1c0f88763 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:48.703545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0b29c-c2dc-4b0f-b3b6-b0204b1a01f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:48.704143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0b29c-c2dc-4b0f-b3b6-b0204b1a01f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:48.704421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0b29c-c2dc-4b0f-b3b6-b0204b1a01f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:48.704694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0b29c-c2dc-4b0f-b3b6-b0204b1a01f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:48.705332 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0b29c-c2dc-4b0f-b3b6-b0204b1a01f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:41:48.705668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0b29c-c2dc-4b0f-b3b6-b0204b1a01f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:48.707263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0b29c-c2dc-4b0f-b3b6-b0204b1a01f0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:48.719531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1313/2584] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:41:48.723729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2da8d7-03bc-411f-ac60-a72e91ad8eb1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:48.729315 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:48.733859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2da8d7-03bc-411f-ac60-a72e91ad8eb1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:48.738584 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682eedf-fdb8-4197-9803-e0139525197e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:48.739175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682eedf-fdb8-4197-9803-e0139525197e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:48.739453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682eedf-fdb8-4197-9803-e0139525197e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:48.739724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682eedf-fdb8-4197-9803-e0139525197e admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:48.740357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682eedf-fdb8-4197-9803-e0139525197e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:48.742293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682eedf-fdb8-4197-9803-e0139525197e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:48.807527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1272/2585] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:41:48 2022] POST /identity/v3/projects => generated 405 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:48.812008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0b29c-c2dc-4b0f-b3b6-b0204b1a01f0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:48.822148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0b29c-c2dc-4b0f-b3b6-b0204b1a01f0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:48.885803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23452e4a-11e4-4921-98a1-3dbb3a2bf603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:48.886133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23452e4a-11e4-4921-98a1-3dbb3a2bf603 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:48.886626 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23452e4a-11e4-4921-98a1-3dbb3a2bf603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:48.886912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23452e4a-11e4-4921-98a1-3dbb3a2bf603 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:48.887536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23452e4a-11e4-4921-98a1-3dbb3a2bf603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:48.889417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23452e4a-11e4-4921-98a1-3dbb3a2bf603 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:48.996415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1314/2586] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:41:48 2022] POST /identity/v3/users => generated 303 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:49.000563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682eedf-fdb8-4197-9803-e0139525197e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:49.011804 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682eedf-fdb8-4197-9803-e0139525197e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:49.016450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ccec6-7e76-4b2e-96ce-9fb535aaadf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:49.017252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ccec6-7e76-4b2e-96ce-9fb535aaadf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:49.017614 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ccec6-7e76-4b2e-96ce-9fb535aaadf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:49.017885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ccec6-7e76-4b2e-96ce-9fb535aaadf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:49.018635 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ccec6-7e76-4b2e-96ce-9fb535aaadf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:49.020228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ccec6-7e76-4b2e-96ce-9fb535aaadf4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:49.031043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1273/2587] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:49.035017 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23452e4a-11e4-4921-98a1-3dbb3a2bf603 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:49.045472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23452e4a-11e4-4921-98a1-3dbb3a2bf603 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:49.050370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321551e-750e-441e-bd46-5459f63e8f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:49.057735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321551e-750e-441e-bd46-5459f63e8f66 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:49.057735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321551e-750e-441e-bd46-5459f63e8f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:49.057735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321551e-750e-441e-bd46-5459f63e8f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:49.057735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321551e-750e-441e-bd46-5459f63e8f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:49.057735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321551e-750e-441e-bd46-5459f63e8f66 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:49.065108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1315/2588] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:49.069749 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ccec6-7e76-4b2e-96ce-9fb535aaadf4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:49.080471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ccec6-7e76-4b2e-96ce-9fb535aaadf4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:49.085077 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac1696-7f0b-4a51-93a7-f5c1543e440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:49.085757 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac1696-7f0b-4a51-93a7-f5c1543e440e admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:49.086034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac1696-7f0b-4a51-93a7-f5c1543e440e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:49.086340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac1696-7f0b-4a51-93a7-f5c1543e440e admin admin] PATH_INFO: `/v3/projects/be5ab5a795e74ffb84e2b83db23a41b2/users/bbe9d70c1cc248a0a18c258553f25f8e/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:49.102050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac1696-7f0b-4a51-93a7-f5c1543e440e admin admin] RBAC: Authorizing `identity:create_grant(project_id=be5ab5a795e74ffb84e2b83db23a41b2, user_id=bbe9d70c1cc248a0a18c258553f25f8e, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:49.104820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac1696-7f0b-4a51-93a7-f5c1543e440e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:49.195998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1274/2589] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:49 2022] PUT /identity/v3/projects/be5ab5a795e74ffb84e2b83db23a41b2/users/bbe9d70c1cc248a0a18c258553f25f8e/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:49.201725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0a62-90e6-4b37-bf54-8747040b19a9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:49.202086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0a62-90e6-4b37-bf54-8747040b19a9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:49.202370 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0a62-90e6-4b37-bf54-8747040b19a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:49.209710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:49.222969 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a90b0a62-90e6-4b37-bf54-8747040b19a9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:49.224186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a90b0a62-90e6-4b37-bf54-8747040b19a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:49.282436 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1316/2590] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:49.292091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac1696-7f0b-4a51-93a7-f5c1543e440e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:49.294050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0cac1696-7f0b-4a51-93a7-f5c1543e440e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:49.354352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac1696-7f0b-4a51-93a7-f5c1543e440e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:49.358810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4f6e-453b-4bf2-bb31-2ce4139f33b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:49.359422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4f6e-453b-4bf2-bb31-2ce4139f33b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:49.359702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4f6e-453b-4bf2-bb31-2ce4139f33b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:49.360006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4f6e-453b-4bf2-bb31-2ce4139f33b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:49.369523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd4f6e-453b-4bf2-bb31-2ce4139f33b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:49.371589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd4f6e-453b-4bf2-bb31-2ce4139f33b1 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:49.381641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1275/2591] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:50.602273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29595b70-41db-4b95-94d5-b0725943e71b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:50.602831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29595b70-41db-4b95-94d5-b0725943e71b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:50.602831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29595b70-41db-4b95-94d5-b0725943e71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:50.631869 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-29595b70-41db-4b95-94d5-b0725943e71b None None] MFA Rules not processed for user `bbe9d70c1cc248a0a18c258553f25f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:50.633140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-29595b70-41db-4b95-94d5-b0725943e71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:50.709834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1317/2592] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:50 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:50.715071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4f6e-453b-4bf2-bb31-2ce4139f33b1 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:50.725834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4f6e-453b-4bf2-bb31-2ce4139f33b1 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:50.730967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c109-a080-42fc-a08d-6a8987bc432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:50.731570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c109-a080-42fc-a08d-6a8987bc432a admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:50.731848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c109-a080-42fc-a08d-6a8987bc432a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:50.732119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c109-a080-42fc-a08d-6a8987bc432a admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:50.732759 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311c109-a080-42fc-a08d-6a8987bc432a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:50.734678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311c109-a080-42fc-a08d-6a8987bc432a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:50.819857 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1276/2593] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:41:50 2022] POST /identity/v3/projects => generated 403 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:50.824756 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29595b70-41db-4b95-94d5-b0725943e71b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:50.835418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29595b70-41db-4b95-94d5-b0725943e71b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:50.892325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822b1bb-007c-461f-9f39-6c45ad4b1f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:50.892923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822b1bb-007c-461f-9f39-6c45ad4b1f2d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:50.893283 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822b1bb-007c-461f-9f39-6c45ad4b1f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:50.893476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822b1bb-007c-461f-9f39-6c45ad4b1f2d admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:50.894119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822b1bb-007c-461f-9f39-6c45ad4b1f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:50.896019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822b1bb-007c-461f-9f39-6c45ad4b1f2d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:50.984327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1318/2594] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:41:50 2022] POST /identity/v3/users => generated 308 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:50.988461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c109-a080-42fc-a08d-6a8987bc432a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:50.999345 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c109-a080-42fc-a08d-6a8987bc432a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:51.003839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d0bcd-03ce-4b6f-affe-344c717400a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:51.004471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d0bcd-03ce-4b6f-affe-344c717400a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:51.004745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d0bcd-03ce-4b6f-affe-344c717400a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:51.005015 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d0bcd-03ce-4b6f-affe-344c717400a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:51.005829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d0bcd-03ce-4b6f-affe-344c717400a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:51.010022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d0bcd-03ce-4b6f-affe-344c717400a6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:51.020208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1277/2595] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:51.028224 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822b1bb-007c-461f-9f39-6c45ad4b1f2d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:51.041491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822b1bb-007c-461f-9f39-6c45ad4b1f2d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:51.046921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fb4cf-3534-4f8a-9edc-9ee4d119a8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:51.047582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fb4cf-3534-4f8a-9edc-9ee4d119a8c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:51.047870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fb4cf-3534-4f8a-9edc-9ee4d119a8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:51.048138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fb4cf-3534-4f8a-9edc-9ee4d119a8c6 admin admin] PATH_INFO: `/v3/projects/8f1a7fee4a56408bb5729943b99eb4f9/users/731daa2316bd4fedb4d2949566e60b03/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:51.063746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362fb4cf-3534-4f8a-9edc-9ee4d119a8c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f1a7fee4a56408bb5729943b99eb4f9, user_id=731daa2316bd4fedb4d2949566e60b03, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:51.066429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362fb4cf-3534-4f8a-9edc-9ee4d119a8c6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:51.150314 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:51.156806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1319/2596] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:51 2022] PUT /identity/v3/projects/8f1a7fee4a56408bb5729943b99eb4f9/users/731daa2316bd4fedb4d2949566e60b03/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:51.160888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d0bcd-03ce-4b6f-affe-344c717400a6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:51.171801 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d0bcd-03ce-4b6f-affe-344c717400a6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:51.229180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846c1f9-1346-4558-af36-32f42d3d55c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:51.230367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846c1f9-1346-4558-af36-32f42d3d55c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:51.230823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846c1f9-1346-4558-af36-32f42d3d55c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:51.231096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846c1f9-1346-4558-af36-32f42d3d55c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:51.231732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846c1f9-1346-4558-af36-32f42d3d55c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:51.233352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846c1f9-1346-4558-af36-32f42d3d55c9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:51.243089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1278/2597] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:51.247797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fb4cf-3534-4f8a-9edc-9ee4d119a8c6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:51.259382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fb4cf-3534-4f8a-9edc-9ee4d119a8c6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:51.265121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624867d0-8a0b-4381-a813-ac948f0e421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:51.265806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624867d0-8a0b-4381-a813-ac948f0e421e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:51.266091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624867d0-8a0b-4381-a813-ac948f0e421e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:51.266367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624867d0-8a0b-4381-a813-ac948f0e421e admin admin] PATH_INFO: `/v3/projects/8f1a7fee4a56408bb5729943b99eb4f9/users/731daa2316bd4fedb4d2949566e60b03/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:51.268295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624867d0-8a0b-4381-a813-ac948f0e421e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f1a7fee4a56408bb5729943b99eb4f9, user_id=731daa2316bd4fedb4d2949566e60b03, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:51.271008 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624867d0-8a0b-4381-a813-ac948f0e421e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:51.282121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1320/2598] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:51 2022] PUT /identity/v3/projects/8f1a7fee4a56408bb5729943b99eb4f9/users/731daa2316bd4fedb4d2949566e60b03/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:52.380810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6c0a0-727d-4bdd-aa4d-23d6f4cd73b1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:52.381314 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6c0a0-727d-4bdd-aa4d-23d6f4cd73b1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:52.381444 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6c0a0-727d-4bdd-aa4d-23d6f4cd73b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:52.408533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:52.411630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fcf6c0a0-727d-4bdd-aa4d-23d6f4cd73b1 None None] MFA Rules not processed for user `731daa2316bd4fedb4d2949566e60b03`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:52.412864 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fcf6c0a0-727d-4bdd-aa4d-23d6f4cd73b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:52.482971 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1279/2599] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:52 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:53.019968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624867d0-8a0b-4381-a813-ac948f0e421e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:53.021737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-624867d0-8a0b-4381-a813-ac948f0e421e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:53.095668 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624867d0-8a0b-4381-a813-ac948f0e421e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:53.100649 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63d25e-a593-4a21-b721-18f19fb89225 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:53.101253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63d25e-a593-4a21-b721-18f19fb89225 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:53.103667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63d25e-a593-4a21-b721-18f19fb89225 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:53.103667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63d25e-a593-4a21-b721-18f19fb89225 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:53.103667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9e63d25e-a593-4a21-b721-18f19fb89225 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:53.115566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63d25e-a593-4a21-b721-18f19fb89225 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:53.117133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63d25e-a593-4a21-b721-18f19fb89225 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:53.129227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1321/2600] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:53.802174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6c0a0-727d-4bdd-aa4d-23d6f4cd73b1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:53.867061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6c0a0-727d-4bdd-aa4d-23d6f4cd73b1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:53.872725 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:53.873351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:53.873660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:53.873934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] PATH_INFO: `/v3/users/bbe9d70c1cc248a0a18c258553f25f8e` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:53.875049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe9d70c1cc248a0a18c258553f25f8e)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:53.876965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:54.060871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe9d70c1cc248a0a18c258553f25f8e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.061449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe9d70c1cc248a0a18c258553f25f8e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.160561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe9d70c1cc248a0a18c258553f25f8e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.174371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe9d70c1cc248a0a18c258553f25f8e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.181113 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe9d70c1cc248a0a18c258553f25f8e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.186984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1280/2601] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:53 2022] DELETE /identity/v3/users/bbe9d70c1cc248a0a18c258553f25f8e => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:54.484701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63d25e-a593-4a21-b721-18f19fb89225 service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:54.493383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63d25e-a593-4a21-b721-18f19fb89225 service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:54.552852 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:54.553568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:54.553850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:54.554119 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] PATH_INFO: `/v3/projects/be5ab5a795e74ffb84e2b83db23a41b2` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:54.555227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be5ab5a795e74ffb84e2b83db23a41b2)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:54.557144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:54.687360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be5ab5a795e74ffb84e2b83db23a41b2', 'request_id': 'req-10bf5726-fe8e-4b41-8793-8724a2b18110'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.738327 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be5ab5a795e74ffb84e2b83db23a41b2', 'request_id': 'req-10bf5726-fe8e-4b41-8793-8724a2b18110'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.745762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be5ab5a795e74ffb84e2b83db23a41b2', 'request_id': 'req-10bf5726-fe8e-4b41-8793-8724a2b18110'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.756203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] The token cache is being invalidate because project be5ab5a795e74ffb84e2b83db23a41b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:54.756491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.757131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.758696 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1322/2602] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:54 2022] DELETE /identity/v3/projects/be5ab5a795e74ffb84e2b83db23a41b2 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:54.763442 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:54.764842 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:54.777812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae22134-fe79-4fe1-9341-94ac8b12210d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:54.789986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:54.835618 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:54.836243 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:54.836524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:54.836798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] PATH_INFO: `/v3/users/731daa2316bd4fedb4d2949566e60b03` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:54.837969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=731daa2316bd4fedb4d2949566e60b03)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:54.840544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:54.974616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '731daa2316bd4fedb4d2949566e60b03'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:54.975187 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '731daa2316bd4fedb4d2949566e60b03'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:55.029305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '731daa2316bd4fedb4d2949566e60b03'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:55.042361 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '731daa2316bd4fedb4d2949566e60b03'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:55.049531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cb0eb344-aadc-42e1-9e76-dcf005dc7df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '731daa2316bd4fedb4d2949566e60b03'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:55.055335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1281/2603] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:54 2022] DELETE /identity/v3/users/731daa2316bd4fedb4d2949566e60b03 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:55.308800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:55.310275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:55.323878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf5726-fe8e-4b41-8793-8724a2b18110 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:55.325030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:55.382066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:55.382705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:55.382984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:55.383256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] PATH_INFO: `/v3/projects/8f1a7fee4a56408bb5729943b99eb4f9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:55.384381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f1a7fee4a56408bb5729943b99eb4f9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:55.386421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:55.500022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f1a7fee4a56408bb5729943b99eb4f9', 'request_id': 'req-896fccae-8fa8-4133-bb2c-8b4235820b70'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:55.567077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f1a7fee4a56408bb5729943b99eb4f9', 'request_id': 'req-896fccae-8fa8-4133-bb2c-8b4235820b70'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:55.574122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f1a7fee4a56408bb5729943b99eb4f9', 'request_id': 'req-896fccae-8fa8-4133-bb2c-8b4235820b70'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:55.584634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] The token cache is being invalidate because project 8f1a7fee4a56408bb5729943b99eb4f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:55.584925 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:55.585599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-896fccae-8fa8-4133-bb2c-8b4235820b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:55.587087 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1323/2604] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:55 2022] DELETE /identity/v3/projects/8f1a7fee4a56408bb5729943b99eb4f9 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:55.592975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad47db-0d74-4016-a984-5a3933d5e0ac None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:55.593300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad47db-0d74-4016-a984-5a3933d5e0ac None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:55.593617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad47db-0d74-4016-a984-5a3933d5e0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:55.614957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d2ad47db-0d74-4016-a984-5a3933d5e0ac None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:55.616190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d2ad47db-0d74-4016-a984-5a3933d5e0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:55.677533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1282/2605] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:55.686149 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5659-a8c6-4178-a4d1-de9fd43913da None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:55.686432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5659-a8c6-4178-a4d1-de9fd43913da None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:55.686724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5659-a8c6-4178-a4d1-de9fd43913da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:55.706899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c63e5659-a8c6-4178-a4d1-de9fd43913da None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:55.708091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c63e5659-a8c6-4178-a4d1-de9fd43913da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:55.716171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1324/2606] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:55.720343 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad47db-0d74-4016-a984-5a3933d5e0ac None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:55.731004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad47db-0d74-4016-a984-5a3933d5e0ac None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:55.735677 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2ee51-4b29-48e2-ac53-5a75eaa741d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:55.736278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2ee51-4b29-48e2-ac53-5a75eaa741d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:55.736554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2ee51-4b29-48e2-ac53-5a75eaa741d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:55.736823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2ee51-4b29-48e2-ac53-5a75eaa741d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:55.737482 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2ee51-4b29-48e2-ac53-5a75eaa741d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:41:55.737788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2ee51-4b29-48e2-ac53-5a75eaa741d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:55.739383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2ee51-4b29-48e2-ac53-5a75eaa741d9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:55.751549 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1283/2607] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:41:55.755754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e5659-a8c6-4178-a4d1-de9fd43913da None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:55.765145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e5659-a8c6-4178-a4d1-de9fd43913da None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:55.769854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9bac5-b7f3-4060-a5c6-474ee5cfdf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:55.770450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9bac5-b7f3-4060-a5c6-474ee5cfdf81 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:55.770728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9bac5-b7f3-4060-a5c6-474ee5cfdf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:55.771001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9bac5-b7f3-4060-a5c6-474ee5cfdf81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:55.771633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9bac5-b7f3-4060-a5c6-474ee5cfdf81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:55.773543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9bac5-b7f3-4060-a5c6-474ee5cfdf81 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:55.880784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1325/2608] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:41:55 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:55.881974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2ee51-4b29-48e2-ac53-5a75eaa741d9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:55.892474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2ee51-4b29-48e2-ac53-5a75eaa741d9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:55.913202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:55.949553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35dbb0-ff84-4293-a54f-f619034326ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:55.950282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35dbb0-ff84-4293-a54f-f619034326ea admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:55.950617 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35dbb0-ff84-4293-a54f-f619034326ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:55.950892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35dbb0-ff84-4293-a54f-f619034326ea admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:55.951520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35dbb0-ff84-4293-a54f-f619034326ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:55.953429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35dbb0-ff84-4293-a54f-f619034326ea admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:56.076819 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1284/2609] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:41:55 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:56.081295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9bac5-b7f3-4060-a5c6-474ee5cfdf81 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:56.093122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9bac5-b7f3-4060-a5c6-474ee5cfdf81 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:56.097842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c315e-9da4-4414-970a-190c4dba4603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:56.098434 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c315e-9da4-4414-970a-190c4dba4603 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:56.098709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c315e-9da4-4414-970a-190c4dba4603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:56.099020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c315e-9da4-4414-970a-190c4dba4603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:56.099652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c315e-9da4-4414-970a-190c4dba4603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:56.101209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c315e-9da4-4414-970a-190c4dba4603 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:56.111546 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1326/2610] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:41:56.115869 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35dbb0-ff84-4293-a54f-f619034326ea admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:56.126645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35dbb0-ff84-4293-a54f-f619034326ea admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:56.131393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103b273-8966-4ccf-b85b-27b6f2c018e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:56.131918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103b273-8966-4ccf-b85b-27b6f2c018e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:56.132192 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103b273-8966-4ccf-b85b-27b6f2c018e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:56.132457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103b273-8966-4ccf-b85b-27b6f2c018e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:56.133087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103b273-8966-4ccf-b85b-27b6f2c018e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:56.134729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103b273-8966-4ccf-b85b-27b6f2c018e3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:56.145939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1285/2611] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:41:56.150763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c315e-9da4-4414-970a-190c4dba4603 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:56.163262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c315e-9da4-4414-970a-190c4dba4603 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:56.168036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec324e-11c9-47d1-a02e-677ee0f0a168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:56.168692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec324e-11c9-47d1-a02e-677ee0f0a168 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:56.168974 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec324e-11c9-47d1-a02e-677ee0f0a168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:56.169247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec324e-11c9-47d1-a02e-677ee0f0a168 admin admin] PATH_INFO: `/v3/projects/ae18d4823ef744eea47fd965824b2133/users/1ee50e8559ef4bf9a19c8355048e9149/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:56.184244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec324e-11c9-47d1-a02e-677ee0f0a168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae18d4823ef744eea47fd965824b2133, user_id=1ee50e8559ef4bf9a19c8355048e9149, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:56.188247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec324e-11c9-47d1-a02e-677ee0f0a168 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:56.287537 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1327/2612] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:56 2022] PUT /identity/v3/projects/ae18d4823ef744eea47fd965824b2133/users/1ee50e8559ef4bf9a19c8355048e9149/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:56.298259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd1ce5-bb8a-4dc7-a5ef-c996bd1b1597 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:56.298615 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd1ce5-bb8a-4dc7-a5ef-c996bd1b1597 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:56.298898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd1ce5-bb8a-4dc7-a5ef-c996bd1b1597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:56.329189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d8fd1ce5-bb8a-4dc7-a5ef-c996bd1b1597 None None] MFA Rules not processed for user `1ee50e8559ef4bf9a19c8355048e9149`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:56.330473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d8fd1ce5-bb8a-4dc7-a5ef-c996bd1b1597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:56.402606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1286/2613] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:56.407950 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec324e-11c9-47d1-a02e-677ee0f0a168 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:56.443472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:56.473021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec324e-11c9-47d1-a02e-677ee0f0a168 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:56.477820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ecf2b-3d60-4d38-beaf-e72eafd30f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:56.478435 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ecf2b-3d60-4d38-beaf-e72eafd30f0e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:56.478718 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ecf2b-3d60-4d38-beaf-e72eafd30f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:56.478993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ecf2b-3d60-4d38-beaf-e72eafd30f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:56.480656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ecf2b-3d60-4d38-beaf-e72eafd30f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:56.484081 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ecf2b-3d60-4d38-beaf-e72eafd30f0e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:56.592899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1328/2614] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:41:56 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:41:56.598202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd1ce5-bb8a-4dc7-a5ef-c996bd1b1597 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:56.612109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd1ce5-bb8a-4dc7-a5ef-c996bd1b1597 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:56.678773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac5120-1a1c-4c51-b06c-3c84049195a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:56.679384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac5120-1a1c-4c51-b06c-3c84049195a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:56.679657 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac5120-1a1c-4c51-b06c-3c84049195a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:56.679925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac5120-1a1c-4c51-b06c-3c84049195a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:56.680553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac5120-1a1c-4c51-b06c-3c84049195a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:56.682497 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac5120-1a1c-4c51-b06c-3c84049195a4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:56.784685 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1287/2615] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:41:56 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:56.788661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ecf2b-3d60-4d38-beaf-e72eafd30f0e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:56.800002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ecf2b-3d60-4d38-beaf-e72eafd30f0e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:56.804635 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b979b-db19-4169-a25c-a8838f61d383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:56.808728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b979b-db19-4169-a25c-a8838f61d383 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:56.808728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b979b-db19-4169-a25c-a8838f61d383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:56.808728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b979b-db19-4169-a25c-a8838f61d383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:56.808728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b979b-db19-4169-a25c-a8838f61d383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:56.808728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b979b-db19-4169-a25c-a8838f61d383 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:56.820024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1329/2616] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:56.824311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac5120-1a1c-4c51-b06c-3c84049195a4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:56.835408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac5120-1a1c-4c51-b06c-3c84049195a4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:56.839978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287589f-3823-494f-9536-66ead915e1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:56.840679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287589f-3823-494f-9536-66ead915e1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:56.841001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287589f-3823-494f-9536-66ead915e1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:56.841382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287589f-3823-494f-9536-66ead915e1bd admin admin] PATH_INFO: `/v3/projects/4c29e963566f402abd9f78bb5977d374/users/8a3bc2f4604d49aaacf78ff123832367/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:56.856816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287589f-3823-494f-9536-66ead915e1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c29e963566f402abd9f78bb5977d374, user_id=8a3bc2f4604d49aaacf78ff123832367, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:56.860693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287589f-3823-494f-9536-66ead915e1bd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:56.944939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:56.951178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1288/2617] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:56 2022] PUT /identity/v3/projects/4c29e963566f402abd9f78bb5977d374/users/8a3bc2f4604d49aaacf78ff123832367/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:56.955374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b979b-db19-4169-a25c-a8838f61d383 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:56.966819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b979b-db19-4169-a25c-a8838f61d383 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:57.024377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ccafd-2f89-4f7c-a081-05d36f801bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:57.024984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ccafd-2f89-4f7c-a081-05d36f801bec admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:57.025266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ccafd-2f89-4f7c-a081-05d36f801bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:57.025574 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ccafd-2f89-4f7c-a081-05d36f801bec admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:57.026213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ccafd-2f89-4f7c-a081-05d36f801bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:57.027806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ccafd-2f89-4f7c-a081-05d36f801bec admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:57.038265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1330/2618] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:41:57.042431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287589f-3823-494f-9536-66ead915e1bd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:57.059663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287589f-3823-494f-9536-66ead915e1bd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:57.059663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d2dec-54d7-40e8-b37b-4cd08c08a2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:57.059663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d2dec-54d7-40e8-b37b-4cd08c08a2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:57.059663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d2dec-54d7-40e8-b37b-4cd08c08a2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:57.060123 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d2dec-54d7-40e8-b37b-4cd08c08a2e4 admin admin] PATH_INFO: `/v3/projects/4c29e963566f402abd9f78bb5977d374/users/8a3bc2f4604d49aaacf78ff123832367/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:57.061447 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d2dec-54d7-40e8-b37b-4cd08c08a2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c29e963566f402abd9f78bb5977d374, user_id=8a3bc2f4604d49aaacf78ff123832367, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:57.064196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d2dec-54d7-40e8-b37b-4cd08c08a2e4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:57.075838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1289/2619] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:57 2022] PUT /identity/v3/projects/4c29e963566f402abd9f78bb5977d374/users/8a3bc2f4604d49aaacf78ff123832367/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:57.087142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20261d85-cbf0-4a89-b79b-372be8bc7b35 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:57.087432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20261d85-cbf0-4a89-b79b-372be8bc7b35 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:57.087709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20261d85-cbf0-4a89-b79b-372be8bc7b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:57.116591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-20261d85-cbf0-4a89-b79b-372be8bc7b35 None None] MFA Rules not processed for user `8a3bc2f4604d49aaacf78ff123832367`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:57.117909 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-20261d85-cbf0-4a89-b79b-372be8bc7b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:57.199800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1331/2620] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:41:57.212234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d2dec-54d7-40e8-b37b-4cd08c08a2e4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:57.214026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2c2d2dec-54d7-40e8-b37b-4cd08c08a2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:57.277943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d2dec-54d7-40e8-b37b-4cd08c08a2e4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:57.281999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01bb07-a08d-43e1-88b9-962d58d9d578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:57.282606 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01bb07-a08d-43e1-88b9-962d58d9d578 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:57.282885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01bb07-a08d-43e1-88b9-962d58d9d578 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:57.283159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01bb07-a08d-43e1-88b9-962d58d9d578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:57.292983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01bb07-a08d-43e1-88b9-962d58d9d578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:57.295035 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01bb07-a08d-43e1-88b9-962d58d9d578 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:57.304997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1290/2621] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:57.338058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20261d85-cbf0-4a89-b79b-372be8bc7b35 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:57.368949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:57.403261 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20261d85-cbf0-4a89-b79b-372be8bc7b35 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:57.408138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:57.408803 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:57.409223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:57.409522 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] PATH_INFO: `/v3/users/1ee50e8559ef4bf9a19c8355048e9149` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:57.410680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee50e8559ef4bf9a19c8355048e9149)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:57.412742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:57.580987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee50e8559ef4bf9a19c8355048e9149'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:57.593221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee50e8559ef4bf9a19c8355048e9149'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:57.600303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee50e8559ef4bf9a19c8355048e9149'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:57.646639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee50e8559ef4bf9a19c8355048e9149'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:57.647254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee50e8559ef4bf9a19c8355048e9149'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:57.653890 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1332/2622] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:57 2022] DELETE /identity/v3/users/1ee50e8559ef4bf9a19c8355048e9149 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:57.658512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabc66f-e685-4f17-9712-cd0db5b97e83 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:57.658799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabc66f-e685-4f17-9712-cd0db5b97e83 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:57.659672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabc66f-e685-4f17-9712-cd0db5b97e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:57.678060 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5aabc66f-e685-4f17-9712-cd0db5b97e83 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:57.679262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5aabc66f-e685-4f17-9712-cd0db5b97e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:57.738541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1291/2623] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:41:57.748686 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:57.750135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:57.806790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebdd510-1af3-4c3b-8976-661663c6cee9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:57.811100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7722ef-7dd4-42e7-a7a8-394880f21734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:57.811738 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7722ef-7dd4-42e7-a7a8-394880f21734 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:57.812019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7722ef-7dd4-42e7-a7a8-394880f21734 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:57.812318 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7722ef-7dd4-42e7-a7a8-394880f21734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:57.821846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7722ef-7dd4-42e7-a7a8-394880f21734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:57.823275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7722ef-7dd4-42e7-a7a8-394880f21734 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:57.832781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1333/2624] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:58.242945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabc66f-e685-4f17-9712-cd0db5b97e83 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:58.244359 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5aabc66f-e685-4f17-9712-cd0db5b97e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:58.264824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabc66f-e685-4f17-9712-cd0db5b97e83 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:58.273290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:58.274396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:58.274842 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:58.275218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] PATH_INFO: `/v3/projects/ae18d4823ef744eea47fd965824b2133` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:58.277365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae18d4823ef744eea47fd965824b2133)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:58.281418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:58.390096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae18d4823ef744eea47fd965824b2133', 'request_id': 'req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:58.458308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae18d4823ef744eea47fd965824b2133', 'request_id': 'req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:58.465254 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae18d4823ef744eea47fd965824b2133', 'request_id': 'req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:58.477852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] The token cache is being invalidate because project ae18d4823ef744eea47fd965824b2133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:58.478144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:58.478806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:58.480318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1292/2625] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:58 2022] DELETE /identity/v3/projects/ae18d4823ef744eea47fd965824b2133 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:58.484510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7722ef-7dd4-42e7-a7a8-394880f21734 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:58.494474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7722ef-7dd4-42e7-a7a8-394880f21734 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:58.544085 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:58.555619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:58.556255 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:58.556538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:58.556811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] PATH_INFO: `/v3/users/8a3bc2f4604d49aaacf78ff123832367` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:58.557992 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a3bc2f4604d49aaacf78ff123832367)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:58.560167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:58.720377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a3bc2f4604d49aaacf78ff123832367'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:58.733080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a3bc2f4604d49aaacf78ff123832367'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:58.740123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a3bc2f4604d49aaacf78ff123832367'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:58.796081 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a3bc2f4604d49aaacf78ff123832367'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:58.796790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-470e2c40-1fd0-42c0-b0fd-fed3e181954d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a3bc2f4604d49aaacf78ff123832367'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:58.804125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1334/2626] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:41:58 2022] DELETE /identity/v3/users/8a3bc2f4604d49aaacf78ff123832367 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:41:59.192048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:59.193510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:59.207063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbe14c-0de9-4589-a8fc-3a1478d47bc1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:59.215756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:59.266502 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:59.267851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:59.268129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:59.268402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] PATH_INFO: `/v3/projects/4c29e963566f402abd9f78bb5977d374` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:59.269548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c29e963566f402abd9f78bb5977d374)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:59.271526 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:59.378507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c29e963566f402abd9f78bb5977d374', 'request_id': 'req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:59.446936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c29e963566f402abd9f78bb5977d374', 'request_id': 'req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:59.453611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c29e963566f402abd9f78bb5977d374', 'request_id': 'req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:59.464092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] The token cache is being invalidate because project 4c29e963566f402abd9f78bb5977d374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:41:59.464382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:59.465008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b45692e6-8728-4b8a-89e8-dcc2e5a6e034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:41:59.467010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1293/2627] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:41:59 2022] DELETE /identity/v3/projects/4c29e963566f402abd9f78bb5977d374 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:41:59.471854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cea315-d418-4f86-b32a-6b2b428d522d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:59.472140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cea315-d418-4f86-b32a-6b2b428d522d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:59.472415 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cea315-d418-4f86-b32a-6b2b428d522d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:59.493616 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a7cea315-d418-4f86-b32a-6b2b428d522d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:59.494813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a7cea315-d418-4f86-b32a-6b2b428d522d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:59.553308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1335/2628] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:59.561059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590770-01be-44bf-be35-b909474e0b47 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:59.561406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590770-01be-44bf-be35-b909474e0b47 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:59.561721 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590770-01be-44bf-be35-b909474e0b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:59.581885 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-57590770-01be-44bf-be35-b909474e0b47 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:41:59.583199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-57590770-01be-44bf-be35-b909474e0b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:41:59.591776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1294/2629] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:41:59.595945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cea315-d418-4f86-b32a-6b2b428d522d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:59.605392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cea315-d418-4f86-b32a-6b2b428d522d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:59.610592 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f68a7-8976-431e-9364-cc5e24fb3850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:59.611180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f68a7-8976-431e-9364-cc5e24fb3850 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:59.611457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f68a7-8976-431e-9364-cc5e24fb3850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:59.611724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f68a7-8976-431e-9364-cc5e24fb3850 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:59.612353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f68a7-8976-431e-9364-cc5e24fb3850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:41:59.612651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f68a7-8976-431e-9364-cc5e24fb3850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:59.614284 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f68a7-8976-431e-9364-cc5e24fb3850 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:59.626733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1336/2630] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:41:59.630884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57590770-01be-44bf-be35-b909474e0b47 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:59.640960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57590770-01be-44bf-be35-b909474e0b47 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:59.645667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0df0e3-b789-4307-ab64-0182727ea29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:59.646271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0df0e3-b789-4307-ab64-0182727ea29d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:59.646551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0df0e3-b789-4307-ab64-0182727ea29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:59.646826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0df0e3-b789-4307-ab64-0182727ea29d admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:59.647461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0df0e3-b789-4307-ab64-0182727ea29d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:59.649544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0df0e3-b789-4307-ab64-0182727ea29d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:59.740355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1295/2631] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:41:59 2022] POST /identity/v3/projects => generated 393 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:59.744535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f68a7-8976-431e-9364-cc5e24fb3850 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:59.755048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f68a7-8976-431e-9364-cc5e24fb3850 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:59.805276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:59.812642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8154e2d-36b2-4e4a-b522-41e4b873f097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:59.813241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154e2d-36b2-4e4a-b522-41e4b873f097 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:59.813545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154e2d-36b2-4e4a-b522-41e4b873f097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:59.813820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8154e2d-36b2-4e4a-b522-41e4b873f097 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:59.814449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8154e2d-36b2-4e4a-b522-41e4b873f097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:59.816395 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8154e2d-36b2-4e4a-b522-41e4b873f097 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:59.904728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1337/2632] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:41:59 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:41:59.916758 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0df0e3-b789-4307-ab64-0182727ea29d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:59.927976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0df0e3-b789-4307-ab64-0182727ea29d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:59.932713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59d86a-338e-4053-9053-48f800bbaa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:59.933307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59d86a-338e-4053-9053-48f800bbaa90 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:59.933709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59d86a-338e-4053-9053-48f800bbaa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:59.933986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59d86a-338e-4053-9053-48f800bbaa90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:59.935367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59d86a-338e-4053-9053-48f800bbaa90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:59.936791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59d86a-338e-4053-9053-48f800bbaa90 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:59.947030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1296/2633] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:41:59.951418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8154e2d-36b2-4e4a-b522-41e4b873f097 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:59.961669 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8154e2d-36b2-4e4a-b522-41e4b873f097 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:41:59.966751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991f224-34eb-4381-a813-b66411010062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:41:59.967341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991f224-34eb-4381-a813-b66411010062 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:41:59.967617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991f224-34eb-4381-a813-b66411010062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:41:59.967886 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991f224-34eb-4381-a813-b66411010062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:41:59.968519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991f224-34eb-4381-a813-b66411010062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:41:59.970152 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991f224-34eb-4381-a813-b66411010062 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:41:59.981029 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1338/2634] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:41:59.986202 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59d86a-338e-4053-9053-48f800bbaa90 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:41:59.996236 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59d86a-338e-4053-9053-48f800bbaa90 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:00.001544 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede17333-f3ac-4577-a492-ff9f22779878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:00.002184 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede17333-f3ac-4577-a492-ff9f22779878 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:00.002459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede17333-f3ac-4577-a492-ff9f22779878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:00.002733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede17333-f3ac-4577-a492-ff9f22779878 admin admin] PATH_INFO: `/v3/projects/ae2d26c45ea74e73a0000ea225892ce0/users/b404c2e3943f419a968957325e76b74e/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:00.017141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede17333-f3ac-4577-a492-ff9f22779878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2d26c45ea74e73a0000ea225892ce0, user_id=b404c2e3943f419a968957325e76b74e, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:00.019829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede17333-f3ac-4577-a492-ff9f22779878 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:00.104145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1297/2635] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:41:59 2022] PUT /identity/v3/projects/ae2d26c45ea74e73a0000ea225892ce0/users/b404c2e3943f419a968957325e76b74e/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:00.115299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d75660-726e-47e2-a32f-5fc561f4c292 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:00.115697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d75660-726e-47e2-a32f-5fc561f4c292 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:00.116001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d75660-726e-47e2-a32f-5fc561f4c292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:00.145011 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b7d75660-726e-47e2-a32f-5fc561f4c292 None None] MFA Rules not processed for user `b404c2e3943f419a968957325e76b74e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:00.146251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b7d75660-726e-47e2-a32f-5fc561f4c292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:00.212265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1339/2636] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:00.217412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede17333-f3ac-4577-a492-ff9f22779878 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:00.259289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:00.279942 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede17333-f3ac-4577-a492-ff9f22779878 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:00.284793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0320d-d07d-4666-80d8-6f8d9c214118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:00.285390 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0320d-d07d-4666-80d8-6f8d9c214118 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:00.285740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0320d-d07d-4666-80d8-6f8d9c214118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:00.286009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0320d-d07d-4666-80d8-6f8d9c214118 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:00.286632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0320d-d07d-4666-80d8-6f8d9c214118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:00.288517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0320d-d07d-4666-80d8-6f8d9c214118 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:00.380324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1298/2637] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:42:00 2022] POST /identity/v3/projects => generated 391 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:00.384455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d75660-726e-47e2-a32f-5fc561f4c292 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:00.395907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d75660-726e-47e2-a32f-5fc561f4c292 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:00.457184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ad8fb-9bbb-4991-8046-b6be654cc89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:00.457828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ad8fb-9bbb-4991-8046-b6be654cc89d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:00.458108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ad8fb-9bbb-4991-8046-b6be654cc89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:00.458379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ad8fb-9bbb-4991-8046-b6be654cc89d admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:00.459012 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ad8fb-9bbb-4991-8046-b6be654cc89d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:00.460923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ad8fb-9bbb-4991-8046-b6be654cc89d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:00.527596 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1340/2638] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:42:00 2022] POST /identity/v3/users => generated 302 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:42:00.532245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0320d-d07d-4666-80d8-6f8d9c214118 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:00.542792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0320d-d07d-4666-80d8-6f8d9c214118 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:00.548439 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32f156-cdfb-4372-af38-41507f4d883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:00.549040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32f156-cdfb-4372-af38-41507f4d883c admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:00.549318 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32f156-cdfb-4372-af38-41507f4d883c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:00.549618 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32f156-cdfb-4372-af38-41507f4d883c admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:00.550251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32f156-cdfb-4372-af38-41507f4d883c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:00.551921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32f156-cdfb-4372-af38-41507f4d883c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:00.563298 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1299/2639] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:00.567278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ad8fb-9bbb-4991-8046-b6be654cc89d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:00.578347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ad8fb-9bbb-4991-8046-b6be654cc89d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:00.582926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530835a9-7ce4-438d-ab1b-cafbfbfe4469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:00.583566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530835a9-7ce4-438d-ab1b-cafbfbfe4469 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:00.583842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530835a9-7ce4-438d-ab1b-cafbfbfe4469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:00.584140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530835a9-7ce4-438d-ab1b-cafbfbfe4469 admin admin] PATH_INFO: `/v3/projects/ce0c2a8026eb49b9bb51d4fde815c16d/users/63ca494cc2cf4a1abc45f9a728af5d25/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:00.599020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530835a9-7ce4-438d-ab1b-cafbfbfe4469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce0c2a8026eb49b9bb51d4fde815c16d, user_id=63ca494cc2cf4a1abc45f9a728af5d25, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:00.601675 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530835a9-7ce4-438d-ab1b-cafbfbfe4469 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:00.681361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1341/2640] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:00 2022] PUT /identity/v3/projects/ce0c2a8026eb49b9bb51d4fde815c16d/users/63ca494cc2cf4a1abc45f9a728af5d25/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:42:00.685793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32f156-cdfb-4372-af38-41507f4d883c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:00.696298 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32f156-cdfb-4372-af38-41507f4d883c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:00.752527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83206ddd-3435-4ed4-9961-33dcddbaf200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:00.753127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83206ddd-3435-4ed4-9961-33dcddbaf200 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:00.753401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83206ddd-3435-4ed4-9961-33dcddbaf200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:00.753701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83206ddd-3435-4ed4-9961-33dcddbaf200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:00.754327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83206ddd-3435-4ed4-9961-33dcddbaf200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:00.755915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83206ddd-3435-4ed4-9961-33dcddbaf200 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:00.765934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1300/2641] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:00.770118 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530835a9-7ce4-438d-ab1b-cafbfbfe4469 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:00.781173 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530835a9-7ce4-438d-ab1b-cafbfbfe4469 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:00.785774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46106db2-3d63-4307-b4b9-1ae239ac2b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:00.786422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46106db2-3d63-4307-b4b9-1ae239ac2b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:00.786701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46106db2-3d63-4307-b4b9-1ae239ac2b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:00.786975 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46106db2-3d63-4307-b4b9-1ae239ac2b7d admin admin] PATH_INFO: `/v3/projects/ce0c2a8026eb49b9bb51d4fde815c16d/users/63ca494cc2cf4a1abc45f9a728af5d25/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:00.788928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46106db2-3d63-4307-b4b9-1ae239ac2b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce0c2a8026eb49b9bb51d4fde815c16d, user_id=63ca494cc2cf4a1abc45f9a728af5d25, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:00.791633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46106db2-3d63-4307-b4b9-1ae239ac2b7d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:00.801904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1342/2642] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:00 2022] PUT /identity/v3/projects/ce0c2a8026eb49b9bb51d4fde815c16d/users/63ca494cc2cf4a1abc45f9a728af5d25/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:00.812964 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af558dd-5522-4c16-a22a-72e83d30605c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:00.813249 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af558dd-5522-4c16-a22a-72e83d30605c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:00.813566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af558dd-5522-4c16-a22a-72e83d30605c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:00.842333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1af558dd-5522-4c16-a22a-72e83d30605c None None] MFA Rules not processed for user `63ca494cc2cf4a1abc45f9a728af5d25`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:00.843554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1af558dd-5522-4c16-a22a-72e83d30605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:00.912280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1301/2643] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:00 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:00.923943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46106db2-3d63-4307-b4b9-1ae239ac2b7d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:00.925900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-46106db2-3d63-4307-b4b9-1ae239ac2b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:00.947533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:01.115549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46106db2-3d63-4307-b4b9-1ae239ac2b7d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:01.119551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f029a-7d5f-4ca3-9631-407e3dd44793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:01.120273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f029a-7d5f-4ca3-9631-407e3dd44793 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:01.120557 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f029a-7d5f-4ca3-9631-407e3dd44793 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:01.120830 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f029a-7d5f-4ca3-9631-407e3dd44793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:01.131844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f029a-7d5f-4ca3-9631-407e3dd44793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:01.133277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f029a-7d5f-4ca3-9631-407e3dd44793 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:01.143986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1343/2644] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:00 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:01.198071 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af558dd-5522-4c16-a22a-72e83d30605c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:01.208664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af558dd-5522-4c16-a22a-72e83d30605c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:01.212251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1da324-6b2c-4b2d-a991-9fda11f17d91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:01.212851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1da324-6b2c-4b2d-a991-9fda11f17d91 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:01.213119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1da324-6b2c-4b2d-a991-9fda11f17d91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:01.213375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1da324-6b2c-4b2d-a991-9fda11f17d91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:01.243679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:01.270709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1da324-6b2c-4b2d-a991-9fda11f17d91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:01.272128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1da324-6b2c-4b2d-a991-9fda11f17d91 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:01.281313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1302/2645] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:06.839901 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f029a-7d5f-4ca3-9631-407e3dd44793 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:06.906778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f029a-7d5f-4ca3-9631-407e3dd44793 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:06.911084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:06.911706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:06.911980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:06.912248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] PATH_INFO: `/v3/users/b404c2e3943f419a968957325e76b74e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:06.913345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b404c2e3943f419a968957325e76b74e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:06.915497 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:07.089666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b404c2e3943f419a968957325e76b74e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:07.102941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b404c2e3943f419a968957325e76b74e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:07.114553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b404c2e3943f419a968957325e76b74e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:07.195505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b404c2e3943f419a968957325e76b74e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:07.196174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b404c2e3943f419a968957325e76b74e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:07.204290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1344/2646] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:42:06 2022] DELETE /identity/v3/users/b404c2e3943f419a968957325e76b74e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:42:07.209501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d194279-bba5-463d-9879-eb7b2c136a73 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:07.210240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d194279-bba5-463d-9879-eb7b2c136a73 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:07.210240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d194279-bba5-463d-9879-eb7b2c136a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:07.230176 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4d194279-bba5-463d-9879-eb7b2c136a73 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:07.232033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4d194279-bba5-463d-9879-eb7b2c136a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:07.293181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1303/2647] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:07.304827 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:07.306419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:07.349501 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:07.365582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfea644-09ff-4c11-9bb4-75069a93bbe7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:07.369161 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a3dab-8dd5-4de9-92c5-1a7c903f5d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:07.369806 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a3dab-8dd5-4de9-92c5-1a7c903f5d4d service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:07.370085 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a3dab-8dd5-4de9-92c5-1a7c903f5d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:07.370455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a3dab-8dd5-4de9-92c5-1a7c903f5d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:07.380154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a3dab-8dd5-4de9-92c5-1a7c903f5d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:07.381620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a3dab-8dd5-4de9-92c5-1a7c903f5d4d service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:07.391707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1345/2648] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:07.797968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d194279-bba5-463d-9879-eb7b2c136a73 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:07.799398 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4d194279-bba5-463d-9879-eb7b2c136a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:07.812595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d194279-bba5-463d-9879-eb7b2c136a73 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:07.818760 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:07.819656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:07.819978 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:07.820252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] PATH_INFO: `/v3/projects/ae2d26c45ea74e73a0000ea225892ce0` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:07.821710 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2d26c45ea74e73a0000ea225892ce0)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:07.824011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:07.924800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2d26c45ea74e73a0000ea225892ce0', 'request_id': 'req-3eeeb069-c730-430b-ac6f-91048f683af9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:07.989663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2d26c45ea74e73a0000ea225892ce0', 'request_id': 'req-3eeeb069-c730-430b-ac6f-91048f683af9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:07.996662 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2d26c45ea74e73a0000ea225892ce0', 'request_id': 'req-3eeeb069-c730-430b-ac6f-91048f683af9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:08.010640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] The token cache is being invalidate because project ae2d26c45ea74e73a0000ea225892ce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:42:08.010929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:08.011594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:08.013713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1304/2649] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:42:07 2022] DELETE /identity/v3/projects/ae2d26c45ea74e73a0000ea225892ce0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:08.017854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a3dab-8dd5-4de9-92c5-1a7c903f5d4d service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:08.027552 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a3dab-8dd5-4de9-92c5-1a7c903f5d4d service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:08.087687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:08.088316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:08.088590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:08.088860 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] PATH_INFO: `/v3/users/63ca494cc2cf4a1abc45f9a728af5d25` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:08.089987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ca494cc2cf4a1abc45f9a728af5d25)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:08.092212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:08.267629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ca494cc2cf4a1abc45f9a728af5d25'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:08.279383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ca494cc2cf4a1abc45f9a728af5d25'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:08.286568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ca494cc2cf4a1abc45f9a728af5d25'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:08.344520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ca494cc2cf4a1abc45f9a728af5d25'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:08.345205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0f61c85-d226-4eca-b1cf-167312c00f55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ca494cc2cf4a1abc45f9a728af5d25'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:08.351863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1346/2650] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:42:08 2022] DELETE /identity/v3/users/63ca494cc2cf4a1abc45f9a728af5d25 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:08.746078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:08.747773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:08.761245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeb069-c730-430b-ac6f-91048f683af9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:08.813326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:08.820504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:08.821203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:08.821505 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:08.823755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] PATH_INFO: `/v3/projects/ce0c2a8026eb49b9bb51d4fde815c16d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:08.823755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce0c2a8026eb49b9bb51d4fde815c16d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:08.825474 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:08.933362 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce0c2a8026eb49b9bb51d4fde815c16d', 'request_id': 'req-18bc4d48-b736-42b3-9674-8aa531399911'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:09.018968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce0c2a8026eb49b9bb51d4fde815c16d', 'request_id': 'req-18bc4d48-b736-42b3-9674-8aa531399911'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:09.026726 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce0c2a8026eb49b9bb51d4fde815c16d', 'request_id': 'req-18bc4d48-b736-42b3-9674-8aa531399911'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:09.038073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] The token cache is being invalidate because project ce0c2a8026eb49b9bb51d4fde815c16d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:42:09.038360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:09.039004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-18bc4d48-b736-42b3-9674-8aa531399911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:09.040531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1305/2651] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:42:08 2022] DELETE /identity/v3/projects/ce0c2a8026eb49b9bb51d4fde815c16d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:09.045353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d09caa-582a-45d5-bf4e-9469f1511c01 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:09.045893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d09caa-582a-45d5-bf4e-9469f1511c01 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:09.046002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d09caa-582a-45d5-bf4e-9469f1511c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:09.065814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c2d09caa-582a-45d5-bf4e-9469f1511c01 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:09.067239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c2d09caa-582a-45d5-bf4e-9469f1511c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:09.122743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:09.130844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1347/2652] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:09.139524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05549e0a-6521-4361-a261-aa88c1f1493f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:09.139810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05549e0a-6521-4361-a261-aa88c1f1493f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:09.140087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05549e0a-6521-4361-a261-aa88c1f1493f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:09.163116 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-05549e0a-6521-4361-a261-aa88c1f1493f None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:09.164336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-05549e0a-6521-4361-a261-aa88c1f1493f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:09.172486 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1306/2653] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:09.176855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d09caa-582a-45d5-bf4e-9469f1511c01 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:09.186503 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d09caa-582a-45d5-bf4e-9469f1511c01 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:09.191120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1b690-cc2e-42c1-9eaa-88082d9cfedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:09.192009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b690-cc2e-42c1-9eaa-88082d9cfedc admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:09.192392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b690-cc2e-42c1-9eaa-88082d9cfedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:09.192749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b690-cc2e-42c1-9eaa-88082d9cfedc admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:09.193519 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1b690-cc2e-42c1-9eaa-88082d9cfedc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:42:09.193976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1b690-cc2e-42c1-9eaa-88082d9cfedc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:09.196204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1b690-cc2e-42c1-9eaa-88082d9cfedc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:09.212055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1348/2654] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:42:09.217514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05549e0a-6521-4361-a261-aa88c1f1493f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:09.226694 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05549e0a-6521-4361-a261-aa88c1f1493f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:09.231322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca97dd4-b10b-4e28-85be-b5707e7d8536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:09.232100 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca97dd4-b10b-4e28-85be-b5707e7d8536 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:09.232373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca97dd4-b10b-4e28-85be-b5707e7d8536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:09.232638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca97dd4-b10b-4e28-85be-b5707e7d8536 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:09.233255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca97dd4-b10b-4e28-85be-b5707e7d8536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:09.235167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca97dd4-b10b-4e28-85be-b5707e7d8536 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:09.312009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1307/2655] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:42:09 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:09.316355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1b690-cc2e-42c1-9eaa-88082d9cfedc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:09.327126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1b690-cc2e-42c1-9eaa-88082d9cfedc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:09.384834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916df742-9ac7-4999-a1cf-d97bfa4027fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:09.385792 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916df742-9ac7-4999-a1cf-d97bfa4027fa admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:09.386077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916df742-9ac7-4999-a1cf-d97bfa4027fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:09.386349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916df742-9ac7-4999-a1cf-d97bfa4027fa admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:09.387037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916df742-9ac7-4999-a1cf-d97bfa4027fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:09.389619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916df742-9ac7-4999-a1cf-d97bfa4027fa admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:09.484402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1349/2656] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:42:09 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:09.488507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca97dd4-b10b-4e28-85be-b5707e7d8536 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:09.499111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca97dd4-b10b-4e28-85be-b5707e7d8536 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:09.503696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc62ce7-575a-44a3-829a-4432d590d2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:09.504289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62ce7-575a-44a3-829a-4432d590d2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:09.504570 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62ce7-575a-44a3-829a-4432d590d2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:09.504848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62ce7-575a-44a3-829a-4432d590d2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:09.505507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc62ce7-575a-44a3-829a-4432d590d2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:09.507102 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc62ce7-575a-44a3-829a-4432d590d2b8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:09.517658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1308/2657] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:09.521867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916df742-9ac7-4999-a1cf-d97bfa4027fa admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:09.533226 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916df742-9ac7-4999-a1cf-d97bfa4027fa admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:09.537921 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5002e7-b299-48ce-94c2-2f8ed2e1b10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:09.538524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5002e7-b299-48ce-94c2-2f8ed2e1b10d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:09.538797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5002e7-b299-48ce-94c2-2f8ed2e1b10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:09.539066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5002e7-b299-48ce-94c2-2f8ed2e1b10d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:09.539701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5002e7-b299-48ce-94c2-2f8ed2e1b10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:09.541279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5002e7-b299-48ce-94c2-2f8ed2e1b10d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:09.551865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1350/2658] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:42:09.556146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc62ce7-575a-44a3-829a-4432d590d2b8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:09.566932 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc62ce7-575a-44a3-829a-4432d590d2b8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:09.573834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c017f4-c079-48d5-8cbf-65e8f6a7f2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:09.574610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c017f4-c079-48d5-8cbf-65e8f6a7f2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:09.575445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c017f4-c079-48d5-8cbf-65e8f6a7f2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:09.575784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c017f4-c079-48d5-8cbf-65e8f6a7f2dc admin admin] PATH_INFO: `/v3/projects/c251d145916f4e8eb2d176140c1f9345/users/9a084bb04fa146fbb99a4ea0af850104/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:09.592756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c017f4-c079-48d5-8cbf-65e8f6a7f2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c251d145916f4e8eb2d176140c1f9345, user_id=9a084bb04fa146fbb99a4ea0af850104, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:09.595548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c017f4-c079-48d5-8cbf-65e8f6a7f2dc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:09.654965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1309/2659] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:09 2022] PUT /identity/v3/projects/c251d145916f4e8eb2d176140c1f9345/users/9a084bb04fa146fbb99a4ea0af850104/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:09.666913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00a720-239b-4913-ba1d-4d77876a6528 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:09.667200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00a720-239b-4913-ba1d-4d77876a6528 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:09.667475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00a720-239b-4913-ba1d-4d77876a6528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:09.695323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-be00a720-239b-4913-ba1d-4d77876a6528 None None] MFA Rules not processed for user `9a084bb04fa146fbb99a4ea0af850104`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:09.696529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-be00a720-239b-4913-ba1d-4d77876a6528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:09.766548 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1351/2660] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:09.777620 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c017f4-c079-48d5-8cbf-65e8f6a7f2dc admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:09.780058 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-35c017f4-c079-48d5-8cbf-65e8f6a7f2dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:09.841126 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c017f4-c079-48d5-8cbf-65e8f6a7f2dc admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:09.846953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c6454-9769-412b-aff5-574d041e3929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:09.847577 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c6454-9769-412b-aff5-574d041e3929 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:09.847848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c6454-9769-412b-aff5-574d041e3929 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:09.848115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c6454-9769-412b-aff5-574d041e3929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:09.857145 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c6454-9769-412b-aff5-574d041e3929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:09.859087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c6454-9769-412b-aff5-574d041e3929 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:09.868334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1310/2661] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:19.762674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:19.863868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:40.070768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00a720-239b-4913-ba1d-4d77876a6528 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:40.070768 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-be00a720-239b-4913-ba1d-4d77876a6528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:40.137576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00a720-239b-4913-ba1d-4d77876a6528 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:40.142127 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:40.142820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:40.143109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:40.143367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] PATH_INFO: `/v3/users/aae99e0f71874495bef6660a6419e676` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:40.144495 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae99e0f71874495bef6660a6419e676)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:40.146458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:40.317186 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae99e0f71874495bef6660a6419e676'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:40.329092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae99e0f71874495bef6660a6419e676'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:40.335868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae99e0f71874495bef6660a6419e676'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:40.417583 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae99e0f71874495bef6660a6419e676'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:40.418209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae99e0f71874495bef6660a6419e676'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:40.424910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1352/2662] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:42:40 2022] DELETE /identity/v3/users/aae99e0f71874495bef6660a6419e676 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:40.429501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d23b27-b0a1-4ceb-98d9-ff49df3265a3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:40.430352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d23b27-b0a1-4ceb-98d9-ff49df3265a3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:40.430352 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d23b27-b0a1-4ceb-98d9-ff49df3265a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:40.449083 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-66d23b27-b0a1-4ceb-98d9-ff49df3265a3 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:40.450307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-66d23b27-b0a1-4ceb-98d9-ff49df3265a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:40.509207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1311/2663] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:40.518996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:40.520448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:40.580215 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a30f8-ca9d-4bf6-93d7-5d54cc5ca0fc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:40.584538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd59cdf-1a5d-4fe2-979e-540ac5e1f015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:40.585148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd59cdf-1a5d-4fe2-979e-540ac5e1f015 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:40.585429 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd59cdf-1a5d-4fe2-979e-540ac5e1f015 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:40.585734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd59cdf-1a5d-4fe2-979e-540ac5e1f015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:40.594816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd59cdf-1a5d-4fe2-979e-540ac5e1f015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:40.596231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd59cdf-1a5d-4fe2-979e-540ac5e1f015 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:40.606213 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1353/2664] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:41.030999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d23b27-b0a1-4ceb-98d9-ff49df3265a3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:41.032941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-66d23b27-b0a1-4ceb-98d9-ff49df3265a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:41.046659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d23b27-b0a1-4ceb-98d9-ff49df3265a3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:41.053164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:41.053974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:41.054148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:41.054479 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] PATH_INFO: `/v3/projects/0d8c07e4bf2b4e779e8dac2e372ab6df` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:41.056297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8c07e4bf2b4e779e8dac2e372ab6df)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:41.058527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:41.167033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8c07e4bf2b4e779e8dac2e372ab6df', 'request_id': 'req-7c7c4219-2de9-4323-b1ca-ef83ab109267'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:41.245909 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8c07e4bf2b4e779e8dac2e372ab6df', 'request_id': 'req-7c7c4219-2de9-4323-b1ca-ef83ab109267'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:41.252777 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8c07e4bf2b4e779e8dac2e372ab6df', 'request_id': 'req-7c7c4219-2de9-4323-b1ca-ef83ab109267'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:41.264209 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] The token cache is being invalidate because project 0d8c07e4bf2b4e779e8dac2e372ab6df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:42:41.264502 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:41.265143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7c7c4219-2de9-4323-b1ca-ef83ab109267 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:41.266642 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1312/2665] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:42:41 2022] DELETE /identity/v3/projects/0d8c07e4bf2b4e779e8dac2e372ab6df => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:41.272561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592987a0-1462-4f6f-91d6-e703415e911e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:41.273123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592987a0-1462-4f6f-91d6-e703415e911e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:41.273123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592987a0-1462-4f6f-91d6-e703415e911e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:41.293142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-592987a0-1462-4f6f-91d6-e703415e911e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:41.294374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-592987a0-1462-4f6f-91d6-e703415e911e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:41.311715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:41.352916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1354/2666] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:41.362152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5558cc-852e-40b5-add6-d14fecbe0fdf None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:41.362436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5558cc-852e-40b5-add6-d14fecbe0fdf None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:41.362709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5558cc-852e-40b5-add6-d14fecbe0fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:41.383753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ca5558cc-852e-40b5-add6-d14fecbe0fdf None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:41.384947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ca5558cc-852e-40b5-add6-d14fecbe0fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:41.392840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1313/2667] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:41.397208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592987a0-1462-4f6f-91d6-e703415e911e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:41.406785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592987a0-1462-4f6f-91d6-e703415e911e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:41.411558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fe6c4-2707-4a19-a79e-c6d1b5b8c283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:41.412154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fe6c4-2707-4a19-a79e-c6d1b5b8c283 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:41.412432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fe6c4-2707-4a19-a79e-c6d1b5b8c283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:41.412701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fe6c4-2707-4a19-a79e-c6d1b5b8c283 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:41.413339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fe6c4-2707-4a19-a79e-c6d1b5b8c283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:42:41.413677 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fe6c4-2707-4a19-a79e-c6d1b5b8c283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:41.415251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fe6c4-2707-4a19-a79e-c6d1b5b8c283 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:41.428196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1355/2668] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:42:41.431609 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5558cc-852e-40b5-add6-d14fecbe0fdf None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:41.441524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5558cc-852e-40b5-add6-d14fecbe0fdf None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:41.446613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65549356-1525-4b5e-8289-a50b5ae0ccbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:41.451610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65549356-1525-4b5e-8289-a50b5ae0ccbd admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:41.451610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65549356-1525-4b5e-8289-a50b5ae0ccbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:41.451610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65549356-1525-4b5e-8289-a50b5ae0ccbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:41.451610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65549356-1525-4b5e-8289-a50b5ae0ccbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:41.451610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65549356-1525-4b5e-8289-a50b5ae0ccbd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:41.522407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1314/2669] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:42:41 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:42:41.526549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fe6c4-2707-4a19-a79e-c6d1b5b8c283 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:41.536606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fe6c4-2707-4a19-a79e-c6d1b5b8c283 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:41.592457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f0a16-e50e-4531-a1f4-5d8020d64752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:41.593058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f0a16-e50e-4531-a1f4-5d8020d64752 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:41.593335 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f0a16-e50e-4531-a1f4-5d8020d64752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:41.593647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f0a16-e50e-4531-a1f4-5d8020d64752 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:41.594277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f0a16-e50e-4531-a1f4-5d8020d64752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:41.596184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f0a16-e50e-4531-a1f4-5d8020d64752 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:41.668740 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1356/2670] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:42:41 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:41.672808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65549356-1525-4b5e-8289-a50b5ae0ccbd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:41.683786 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65549356-1525-4b5e-8289-a50b5ae0ccbd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:41.688267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86560820-4417-406c-9b06-2e606c8ee813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:41.688851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86560820-4417-406c-9b06-2e606c8ee813 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:41.689125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86560820-4417-406c-9b06-2e606c8ee813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:41.689393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86560820-4417-406c-9b06-2e606c8ee813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:41.690059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86560820-4417-406c-9b06-2e606c8ee813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:41.691809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86560820-4417-406c-9b06-2e606c8ee813 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:41.701922 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1315/2671] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:41.705767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f0a16-e50e-4531-a1f4-5d8020d64752 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:41.715800 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f0a16-e50e-4531-a1f4-5d8020d64752 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:41.720384 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da894472-21bf-4a62-964f-be7301ef1a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:41.720979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da894472-21bf-4a62-964f-be7301ef1a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:41.721263 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da894472-21bf-4a62-964f-be7301ef1a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:41.721570 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da894472-21bf-4a62-964f-be7301ef1a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:41.722750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da894472-21bf-4a62-964f-be7301ef1a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:41.724423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da894472-21bf-4a62-964f-be7301ef1a17 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:41.735023 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1357/2672] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:41.739877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86560820-4417-406c-9b06-2e606c8ee813 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:41.749873 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86560820-4417-406c-9b06-2e606c8ee813 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:41.754679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54a03e-dcd8-4195-9df2-96b68d5f0008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:41.755316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54a03e-dcd8-4195-9df2-96b68d5f0008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:41.755594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54a03e-dcd8-4195-9df2-96b68d5f0008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:41.755867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54a03e-dcd8-4195-9df2-96b68d5f0008 admin admin] PATH_INFO: `/v3/projects/138f37da900443bea3ab358845a7c773/users/45e1b0901d174b10b49709d74d5fe5d6/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:41.770119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad54a03e-dcd8-4195-9df2-96b68d5f0008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=138f37da900443bea3ab358845a7c773, user_id=45e1b0901d174b10b49709d74d5fe5d6, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:41.772800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad54a03e-dcd8-4195-9df2-96b68d5f0008 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:41.840855 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1316/2673] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:41 2022] PUT /identity/v3/projects/138f37da900443bea3ab358845a7c773/users/45e1b0901d174b10b49709d74d5fe5d6/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:42:41.845891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbed57-4f0e-4b08-b1d6-a0104eb68566 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:41.850323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbed57-4f0e-4b08-b1d6-a0104eb68566 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:41.850323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbed57-4f0e-4b08-b1d6-a0104eb68566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:41.866300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-56fbed57-4f0e-4b08-b1d6-a0104eb68566 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:41.867502 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-56fbed57-4f0e-4b08-b1d6-a0104eb68566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:41.929159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1358/2674] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:41.940038 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54a03e-dcd8-4195-9df2-96b68d5f0008 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:41.941843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ad54a03e-dcd8-4195-9df2-96b68d5f0008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:41.965573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:42.002764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54a03e-dcd8-4195-9df2-96b68d5f0008 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:42.006644 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2349c-0c49-4fd9-a850-74cbe30bb1e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:42.007251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2349c-0c49-4fd9-a850-74cbe30bb1e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:42.007533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2349c-0c49-4fd9-a850-74cbe30bb1e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:42.007805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2349c-0c49-4fd9-a850-74cbe30bb1e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:42.020028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2349c-0c49-4fd9-a850-74cbe30bb1e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:42.020341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2349c-0c49-4fd9-a850-74cbe30bb1e2 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:42.030736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1317/2675] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:42.070241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbed57-4f0e-4b08-b1d6-a0104eb68566 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:42.071786 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-56fbed57-4f0e-4b08-b1d6-a0104eb68566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:42.084712 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbed57-4f0e-4b08-b1d6-a0104eb68566 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:42.089404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:42.090068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:42.090362 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:42.090692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] PATH_INFO: `/v3/users/9a084bb04fa146fbb99a4ea0af850104` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:42.091833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a084bb04fa146fbb99a4ea0af850104)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:42.093745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:42.117423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:42.317569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a084bb04fa146fbb99a4ea0af850104'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:42.332728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a084bb04fa146fbb99a4ea0af850104'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:42.339749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a084bb04fa146fbb99a4ea0af850104'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:42.417804 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a084bb04fa146fbb99a4ea0af850104'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:42.418473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a084bb04fa146fbb99a4ea0af850104'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:42.424679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1359/2676] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:42:42 2022] DELETE /identity/v3/users/9a084bb04fa146fbb99a4ea0af850104 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:42.429598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9f1e7-59f8-455d-9f6e-934320c17882 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:42.429598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9f1e7-59f8-455d-9f6e-934320c17882 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:42.429835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9f1e7-59f8-455d-9f6e-934320c17882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:42.449766 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-53e9f1e7-59f8-455d-9f6e-934320c17882 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:42.450984 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-53e9f1e7-59f8-455d-9f6e-934320c17882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:42.512838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1318/2677] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:42.522545 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:42.523985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:42.590605 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598e4de-79dc-4e59-b0dd-f7ba9f9ae85a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:42.596952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3ff41-0b83-45d9-8b63-b1b7361bd637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:42.596952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3ff41-0b83-45d9-8b63-b1b7361bd637 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:42.596952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3ff41-0b83-45d9-8b63-b1b7361bd637 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:42.596952 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3ff41-0b83-45d9-8b63-b1b7361bd637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:42.606737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3ff41-0b83-45d9-8b63-b1b7361bd637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:42.609200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3ff41-0b83-45d9-8b63-b1b7361bd637 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:42.621834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1360/2678] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:43.036888 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9f1e7-59f8-455d-9f6e-934320c17882 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:43.038736 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-53e9f1e7-59f8-455d-9f6e-934320c17882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:43.053359 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9f1e7-59f8-455d-9f6e-934320c17882 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:43.058321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:43.059524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.059524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.059737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] PATH_INFO: `/v3/projects/c251d145916f4e8eb2d176140c1f9345` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.060927 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c251d145916f4e8eb2d176140c1f9345)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:43.063761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:43.193153 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c251d145916f4e8eb2d176140c1f9345', 'request_id': 'req-b4137235-1b7f-430b-bf0e-76b6ce71a59b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.247850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c251d145916f4e8eb2d176140c1f9345', 'request_id': 'req-b4137235-1b7f-430b-bf0e-76b6ce71a59b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.262868 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c251d145916f4e8eb2d176140c1f9345', 'request_id': 'req-b4137235-1b7f-430b-bf0e-76b6ce71a59b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.285354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] The token cache is being invalidate because project c251d145916f4e8eb2d176140c1f9345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:42:43.285896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.286959 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b4137235-1b7f-430b-bf0e-76b6ce71a59b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.293513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1319/2679] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:42:43 2022] DELETE /identity/v3/projects/c251d145916f4e8eb2d176140c1f9345 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:43.297279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628e20-1fbd-47f2-982b-a6687da17416 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.297680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628e20-1fbd-47f2-982b-a6687da17416 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.297823 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628e20-1fbd-47f2-982b-a6687da17416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.317854 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-31628e20-1fbd-47f2-982b-a6687da17416 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:43.319073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-31628e20-1fbd-47f2-982b-a6687da17416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:43.399893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1361/2680] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:43.407910 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b0551-688e-4451-9c6f-5043eb773e17 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.420375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b0551-688e-4451-9c6f-5043eb773e17 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.420375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b0551-688e-4451-9c6f-5043eb773e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.454848 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-651b0551-688e-4451-9c6f-5043eb773e17 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:43.456119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-651b0551-688e-4451-9c6f-5043eb773e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:43.472280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31628e20-1fbd-47f2-982b-a6687da17416 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:43.477829 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1320/2681] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:43.484153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31628e20-1fbd-47f2-982b-a6687da17416 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:43.582662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290da12-a6d7-4840-98ab-89782e69bbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:43.582662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290da12-a6d7-4840-98ab-89782e69bbad admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.582662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290da12-a6d7-4840-98ab-89782e69bbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.582662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290da12-a6d7-4840-98ab-89782e69bbad admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.583729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290da12-a6d7-4840-98ab-89782e69bbad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:42:43.583729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290da12-a6d7-4840-98ab-89782e69bbad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:43.583729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290da12-a6d7-4840-98ab-89782e69bbad admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:43.583729 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1362/2682] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:42:43.584887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b0551-688e-4451-9c6f-5043eb773e17 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:43.584887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b0551-688e-4451-9c6f-5043eb773e17 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:43.584887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358db0bb-373f-4379-b208-eeaf65045ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:43.584887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358db0bb-373f-4379-b208-eeaf65045ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.585263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358db0bb-373f-4379-b208-eeaf65045ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.585263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358db0bb-373f-4379-b208-eeaf65045ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.585263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358db0bb-373f-4379-b208-eeaf65045ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:43.585263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358db0bb-373f-4379-b208-eeaf65045ee3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:43.671520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1321/2683] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:42:43 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:43.676591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290da12-a6d7-4840-98ab-89782e69bbad admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:43.687065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290da12-a6d7-4840-98ab-89782e69bbad admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:43.717749 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:43.745882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0b86e-eb95-4409-ba74-b4ce4c947d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:43.746567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0b86e-eb95-4409-ba74-b4ce4c947d4c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.746972 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0b86e-eb95-4409-ba74-b4ce4c947d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.747229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0b86e-eb95-4409-ba74-b4ce4c947d4c admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.747866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0b86e-eb95-4409-ba74-b4ce4c947d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:43.749821 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0b86e-eb95-4409-ba74-b4ce4c947d4c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:43.840382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1363/2684] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:42:43 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:43.841155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358db0bb-373f-4379-b208-eeaf65045ee3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:43.853013 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358db0bb-373f-4379-b208-eeaf65045ee3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:43.859229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf33931-9e7c-4243-8076-9e3092ba8b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:43.861524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf33931-9e7c-4243-8076-9e3092ba8b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.861524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf33931-9e7c-4243-8076-9e3092ba8b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.861524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf33931-9e7c-4243-8076-9e3092ba8b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.862040 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf33931-9e7c-4243-8076-9e3092ba8b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:43.864583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf33931-9e7c-4243-8076-9e3092ba8b17 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:43.877513 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1322/2685] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:42:43.881739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0b86e-eb95-4409-ba74-b4ce4c947d4c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:43.892851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0b86e-eb95-4409-ba74-b4ce4c947d4c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:43.898894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b71028-3707-4c64-a478-969e567cb772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:43.899512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b71028-3707-4c64-a478-969e567cb772 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.899783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b71028-3707-4c64-a478-969e567cb772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.900052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b71028-3707-4c64-a478-969e567cb772 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.900721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b71028-3707-4c64-a478-969e567cb772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:43.902478 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b71028-3707-4c64-a478-969e567cb772 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:43.915790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1364/2686] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:43.919756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf33931-9e7c-4243-8076-9e3092ba8b17 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:43.931014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf33931-9e7c-4243-8076-9e3092ba8b17 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:43.936463 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e62e6-8b98-4e6a-9a5d-d87fa9d254b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:43.937793 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e62e6-8b98-4e6a-9a5d-d87fa9d254b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.938256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e62e6-8b98-4e6a-9a5d-d87fa9d254b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.939208 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e62e6-8b98-4e6a-9a5d-d87fa9d254b1 admin admin] PATH_INFO: `/v3/projects/1f39d8d2806b485984009aab93cb6f9d/users/196e0d38ea1d4669a3185cef7edef14a/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.944072 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:43.959585 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e62e6-8b98-4e6a-9a5d-d87fa9d254b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f39d8d2806b485984009aab93cb6f9d, user_id=196e0d38ea1d4669a3185cef7edef14a, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:43.962650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e62e6-8b98-4e6a-9a5d-d87fa9d254b1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:44.052856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1323/2687] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:43 2022] PUT /identity/v3/projects/1f39d8d2806b485984009aab93cb6f9d/users/196e0d38ea1d4669a3185cef7edef14a/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:42:44.058130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644984c7-6aad-4a0b-a19c-b9db111e3fee None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.058587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644984c7-6aad-4a0b-a19c-b9db111e3fee None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.058927 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644984c7-6aad-4a0b-a19c-b9db111e3fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.094460 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-644984c7-6aad-4a0b-a19c-b9db111e3fee None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:44.095704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-644984c7-6aad-4a0b-a19c-b9db111e3fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:44.207102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1365/2688] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:44.213733 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e62e6-8b98-4e6a-9a5d-d87fa9d254b1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:44.215538 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f36e62e6-8b98-4e6a-9a5d-d87fa9d254b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:44.305740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e62e6-8b98-4e6a-9a5d-d87fa9d254b1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:44.310238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172a4a5-194a-421e-8203-0e926b0b1a29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:44.310238 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172a4a5-194a-421e-8203-0e926b0b1a29 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.310519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172a4a5-194a-421e-8203-0e926b0b1a29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.310791 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172a4a5-194a-421e-8203-0e926b0b1a29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.325698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172a4a5-194a-421e-8203-0e926b0b1a29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:44.327156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172a4a5-194a-421e-8203-0e926b0b1a29 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:44.338596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1324/2689] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:42:47.840987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f03a8-e0ae-410d-9124-036ffe7822c7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:47.844378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f03a8-e0ae-410d-9124-036ffe7822c7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:47.844378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f03a8-e0ae-410d-9124-036ffe7822c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:47.911439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-aa2f03a8-e0ae-410d-9124-036ffe7822c7 None None] MFA Rules not processed for user `45e1b0901d174b10b49709d74d5fe5d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:47.912736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-aa2f03a8-e0ae-410d-9124-036ffe7822c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:47.994099 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1366/2690] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:47.995718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172a4a5-194a-421e-8203-0e926b0b1a29 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:47.997412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7172a4a5-194a-421e-8203-0e926b0b1a29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:48.031816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172a4a5-194a-421e-8203-0e926b0b1a29 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:48.048596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb51df1-8df6-4b74-9483-1bfa58044f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:48.049324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb51df1-8df6-4b74-9483-1bfa58044f5e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:48.051068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb51df1-8df6-4b74-9483-1bfa58044f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:48.051597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb51df1-8df6-4b74-9483-1bfa58044f5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:48.054207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb51df1-8df6-4b74-9483-1bfa58044f5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:48.056524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb51df1-8df6-4b74-9483-1bfa58044f5e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:48.221732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1325/2691] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:42:47 2022] POST /identity/v3/projects => generated 371 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:48.226675 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f03a8-e0ae-410d-9124-036ffe7822c7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:48.260444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f03a8-e0ae-410d-9124-036ffe7822c7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:48.345582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:48.394591 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01b2cd-4bbb-4d26-8d0e-2d71a898e3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:48.395883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01b2cd-4bbb-4d26-8d0e-2d71a898e3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:48.396356 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01b2cd-4bbb-4d26-8d0e-2d71a898e3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:48.397520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01b2cd-4bbb-4d26-8d0e-2d71a898e3a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:48.398361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01b2cd-4bbb-4d26-8d0e-2d71a898e3a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:48.401961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01b2cd-4bbb-4d26-8d0e-2d71a898e3a9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:48.509088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1367/2692] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:42:48 2022] POST /identity/v3/users => generated 292 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:48.513166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb51df1-8df6-4b74-9483-1bfa58044f5e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:48.533631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb51df1-8df6-4b74-9483-1bfa58044f5e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:48.540205 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c84b3-1b5c-4ecf-ab36-7036f44012ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:48.541088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c84b3-1b5c-4ecf-ab36-7036f44012ca admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:48.541624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c84b3-1b5c-4ecf-ab36-7036f44012ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:48.542117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c84b3-1b5c-4ecf-ab36-7036f44012ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:48.543001 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c84b3-1b5c-4ecf-ab36-7036f44012ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:48.548260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c84b3-1b5c-4ecf-ab36-7036f44012ca admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:48.571875 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01b2cd-4bbb-4d26-8d0e-2d71a898e3a9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:48.584871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1326/2693] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:48.591397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01b2cd-4bbb-4d26-8d0e-2d71a898e3a9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:48.609014 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559503d3-55df-46d6-b499-8e3cb380b3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:48.609850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559503d3-55df-46d6-b499-8e3cb380b3e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:48.610170 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559503d3-55df-46d6-b499-8e3cb380b3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:48.610416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559503d3-55df-46d6-b499-8e3cb380b3e3 admin admin] PATH_INFO: `/v3/projects/d9cc40feb4764538ba1ea556b8fb2b37/users/830504c4475248dfa8b0bf0f18397c0a/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:48.638772 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559503d3-55df-46d6-b499-8e3cb380b3e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9cc40feb4764538ba1ea556b8fb2b37, user_id=830504c4475248dfa8b0bf0f18397c0a, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:48.642080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559503d3-55df-46d6-b499-8e3cb380b3e3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:48.726236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1368/2694] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:48 2022] PUT /identity/v3/projects/d9cc40feb4764538ba1ea556b8fb2b37/users/830504c4475248dfa8b0bf0f18397c0a/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:48.730781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c84b3-1b5c-4ecf-ab36-7036f44012ca admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:48.747267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c84b3-1b5c-4ecf-ab36-7036f44012ca admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:48.841477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebabea7-6732-4574-8995-4ad1b1d46ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:48.842286 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebabea7-6732-4574-8995-4ad1b1d46ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:48.842658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebabea7-6732-4574-8995-4ad1b1d46ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:48.843091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebabea7-6732-4574-8995-4ad1b1d46ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:48.843976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebabea7-6732-4574-8995-4ad1b1d46ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:48.845715 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebabea7-6732-4574-8995-4ad1b1d46ae1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:48.873220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559503d3-55df-46d6-b499-8e3cb380b3e3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:48.874794 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1327/2695] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:48.899666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559503d3-55df-46d6-b499-8e3cb380b3e3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:48.909205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517bbd2-c1a8-4a49-88cd-27c076e6b6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:48.910188 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517bbd2-c1a8-4a49-88cd-27c076e6b6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:48.910475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517bbd2-c1a8-4a49-88cd-27c076e6b6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:48.910708 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517bbd2-c1a8-4a49-88cd-27c076e6b6d6 admin admin] PATH_INFO: `/v3/projects/d9cc40feb4764538ba1ea556b8fb2b37/users/830504c4475248dfa8b0bf0f18397c0a/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:48.912679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517bbd2-c1a8-4a49-88cd-27c076e6b6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9cc40feb4764538ba1ea556b8fb2b37, user_id=830504c4475248dfa8b0bf0f18397c0a, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:48.916061 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517bbd2-c1a8-4a49-88cd-27c076e6b6d6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:48.937799 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1369/2696] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:48 2022] PUT /identity/v3/projects/d9cc40feb4764538ba1ea556b8fb2b37/users/830504c4475248dfa8b0bf0f18397c0a/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:50.452783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebabea7-6732-4574-8995-4ad1b1d46ae1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:50.456174 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bebabea7-6732-4574-8995-4ad1b1d46ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:50.571469 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:50.673337 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebabea7-6732-4574-8995-4ad1b1d46ae1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:50.858832 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8fc04-398a-48d8-8428-a27082777691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:50.859903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8fc04-398a-48d8-8428-a27082777691 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:50.861415 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8fc04-398a-48d8-8428-a27082777691 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:50.862244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8fc04-398a-48d8-8428-a27082777691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:50.867256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-99f8fc04-398a-48d8-8428-a27082777691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:50.892540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8fc04-398a-48d8-8428-a27082777691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:50.896470 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8fc04-398a-48d8-8428-a27082777691 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:50.910769 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1328/2697] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:53.263754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6aa135-2ea0-426f-a63a-fbc3ad76a96c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:53.264279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6aa135-2ea0-426f-a63a-fbc3ad76a96c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:53.264279 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6aa135-2ea0-426f-a63a-fbc3ad76a96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:53.328866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-eb6aa135-2ea0-426f-a63a-fbc3ad76a96c None None] MFA Rules not processed for user `196e0d38ea1d4669a3185cef7edef14a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:53.330143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eb6aa135-2ea0-426f-a63a-fbc3ad76a96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:53.473573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1370/2698] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:53.478037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8fc04-398a-48d8-8428-a27082777691 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:53.625598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8fc04-398a-48d8-8428-a27082777691 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:53.647787 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb9daf-e5ff-4770-840c-20d7c251c05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:53.648418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb9daf-e5ff-4770-840c-20d7c251c05e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:53.648708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb9daf-e5ff-4770-840c-20d7c251c05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:53.648966 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb9daf-e5ff-4770-840c-20d7c251c05e admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:53.649650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb9daf-e5ff-4770-840c-20d7c251c05e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:53.651602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb9daf-e5ff-4770-840c-20d7c251c05e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:53.811155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1329/2699] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:42:53 2022] POST /identity/v3/projects => generated 383 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:53.815418 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6aa135-2ea0-426f-a63a-fbc3ad76a96c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:53.836251 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6aa135-2ea0-426f-a63a-fbc3ad76a96c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:53.958277 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1db492-6b59-44b6-95bb-f3d87fa98304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:53.959042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1db492-6b59-44b6-95bb-f3d87fa98304 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:53.959383 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1db492-6b59-44b6-95bb-f3d87fa98304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:53.959674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1db492-6b59-44b6-95bb-f3d87fa98304 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:53.961353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1db492-6b59-44b6-95bb-f3d87fa98304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:53.963450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1db492-6b59-44b6-95bb-f3d87fa98304 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:54.109150 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1371/2700] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:42:53 2022] POST /identity/v3/users => generated 298 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:54.109432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb9daf-e5ff-4770-840c-20d7c251c05e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:54.123858 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb9daf-e5ff-4770-840c-20d7c251c05e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:54.139063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc7f52-0fc4-4bc2-a8c8-81694c33b551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:54.139063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7f52-0fc4-4bc2-a8c8-81694c33b551 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:54.139063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7f52-0fc4-4bc2-a8c8-81694c33b551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:54.139063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7f52-0fc4-4bc2-a8c8-81694c33b551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:54.139423 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc7f52-0fc4-4bc2-a8c8-81694c33b551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:54.140425 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc7f52-0fc4-4bc2-a8c8-81694c33b551 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:54.185336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1330/2701] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:54.186530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1db492-6b59-44b6-95bb-f3d87fa98304 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:54.201185 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1db492-6b59-44b6-95bb-f3d87fa98304 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:54.213194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44df25-d85b-49e9-908d-ddf1077e02fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:54.213194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44df25-d85b-49e9-908d-ddf1077e02fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:54.213194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44df25-d85b-49e9-908d-ddf1077e02fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:54.213194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44df25-d85b-49e9-908d-ddf1077e02fe admin admin] PATH_INFO: `/v3/projects/9ba17fb8e83e4648b0428141a5755f05/users/07d81435cf3c4cebb1caaec654586638/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:54.237292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:54.255114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44df25-d85b-49e9-908d-ddf1077e02fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ba17fb8e83e4648b0428141a5755f05, user_id=07d81435cf3c4cebb1caaec654586638, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:54.259999 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44df25-d85b-49e9-908d-ddf1077e02fe admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:54.340866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1372/2702] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:54 2022] PUT /identity/v3/projects/9ba17fb8e83e4648b0428141a5755f05/users/07d81435cf3c4cebb1caaec654586638/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:54.346520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc7f52-0fc4-4bc2-a8c8-81694c33b551 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:54.360553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc7f52-0fc4-4bc2-a8c8-81694c33b551 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:54.436530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:54.454117 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614afe6-fa5a-46bd-bf4e-912961640acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:54.455163 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614afe6-fa5a-46bd-bf4e-912961640acb admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:54.455276 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614afe6-fa5a-46bd-bf4e-912961640acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:54.455475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614afe6-fa5a-46bd-bf4e-912961640acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:54.456190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614afe6-fa5a-46bd-bf4e-912961640acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:54.457917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614afe6-fa5a-46bd-bf4e-912961640acb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:54.476267 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1331/2703] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:54.477966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44df25-d85b-49e9-908d-ddf1077e02fe admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:54.500521 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44df25-d85b-49e9-908d-ddf1077e02fe admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:54.521134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ca4b7-a1f0-45f0-8151-3ab1f58d1460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:54.522979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ca4b7-a1f0-45f0-8151-3ab1f58d1460 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:54.523393 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ca4b7-a1f0-45f0-8151-3ab1f58d1460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:54.524387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ca4b7-a1f0-45f0-8151-3ab1f58d1460 admin admin] PATH_INFO: `/v3/projects/9ba17fb8e83e4648b0428141a5755f05/users/07d81435cf3c4cebb1caaec654586638/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:54.531707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ca4b7-a1f0-45f0-8151-3ab1f58d1460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ba17fb8e83e4648b0428141a5755f05, user_id=07d81435cf3c4cebb1caaec654586638, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:54.539953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ca4b7-a1f0-45f0-8151-3ab1f58d1460 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:54.573933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1373/2704] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:54 2022] PUT /identity/v3/projects/9ba17fb8e83e4648b0428141a5755f05/users/07d81435cf3c4cebb1caaec654586638/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:57.525501 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406022c1-109d-410c-9f9f-bc7c694e410a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:57.526057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406022c1-109d-410c-9f9f-bc7c694e410a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:57.526057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406022c1-109d-410c-9f9f-bc7c694e410a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:57.557216 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-406022c1-109d-410c-9f9f-bc7c694e410a None None] MFA Rules not processed for user `830504c4475248dfa8b0bf0f18397c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:57.558489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-406022c1-109d-410c-9f9f-bc7c694e410a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:57.641939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1332/2705] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:42:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:57.655981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ca4b7-a1f0-45f0-8151-3ab1f58d1460 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:57.658250 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-593ca4b7-a1f0-45f0-8151-3ab1f58d1460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:57.806069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ca4b7-a1f0-45f0-8151-3ab1f58d1460 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:57.829236 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cf947-7783-4ab1-b0fd-e0bb92130add service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:57.830036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf947-7783-4ab1-b0fd-e0bb92130add service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:57.830311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf947-7783-4ab1-b0fd-e0bb92130add service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:57.830627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf947-7783-4ab1-b0fd-e0bb92130add service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:57.902758 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cf947-7783-4ab1-b0fd-e0bb92130add service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:57.904456 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cf947-7783-4ab1-b0fd-e0bb92130add service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:57.915158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1374/2706] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:43:00.060500 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c296f1-37fb-4454-bc3c-ff001923569c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:00.061024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c296f1-37fb-4454-bc3c-ff001923569c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:00.061024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c296f1-37fb-4454-bc3c-ff001923569c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:00.091219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f5c296f1-37fb-4454-bc3c-ff001923569c None None] MFA Rules not processed for user `1aa49a620d0e42b0947215d200fa45a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:43:00.092569 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f5c296f1-37fb-4454-bc3c-ff001923569c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:00.215707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1333/2707] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:43:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:43:01.318121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494de8b2-65d1-4494-966a-e60fd458311c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:01.318692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494de8b2-65d1-4494-966a-e60fd458311c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:01.318692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494de8b2-65d1-4494-966a-e60fd458311c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:01.364093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-494de8b2-65d1-4494-966a-e60fd458311c None None] MFA Rules not processed for user `45e1b0901d174b10b49709d74d5fe5d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:43:01.365427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-494de8b2-65d1-4494-966a-e60fd458311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:01.391051 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1375/2708] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:43:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:43:02.611263 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed7cb5-c45d-4932-b784-31a58f7380f3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:02.612156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed7cb5-c45d-4932-b784-31a58f7380f3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:02.613293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed7cb5-c45d-4932-b784-31a58f7380f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:02.624848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494de8b2-65d1-4494-966a-e60fd458311c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:02.624848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-494de8b2-65d1-4494-966a-e60fd458311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:02.667052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a0ed7cb5-c45d-4932-b784-31a58f7380f3 None None] MFA Rules not processed for user `07d81435cf3c4cebb1caaec654586638`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:43:02.668876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a0ed7cb5-c45d-4932-b784-31a58f7380f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:02.724475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494de8b2-65d1-4494-966a-e60fd458311c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:02.735062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bef0e-7d49-4ccf-891e-cd323bb4fba1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:02.735737 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bef0e-7d49-4ccf-891e-cd323bb4fba1 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:02.736194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bef0e-7d49-4ccf-891e-cd323bb4fba1 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:02.737432 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bef0e-7d49-4ccf-891e-cd323bb4fba1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:02.738823 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-279bef0e-7d49-4ccf-891e-cd323bb4fba1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:02.748691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:02.754039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bef0e-7d49-4ccf-891e-cd323bb4fba1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:02.756126 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bef0e-7d49-4ccf-891e-cd323bb4fba1 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:02.779897 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1376/2709] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:43:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:43:02.793688 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:02.810265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1334/2710] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:43:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:43:02.826819 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bef0e-7d49-4ccf-891e-cd323bb4fba1 service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:02.828746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-279bef0e-7d49-4ccf-891e-cd323bb4fba1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:02.971387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bef0e-7d49-4ccf-891e-cd323bb4fba1 service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:02.976302 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92295b-fc5f-4f7b-8343-4caf198657d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c23597e0da2454abd2a5354d779b092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:02.976937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92295b-fc5f-4f7b-8343-4caf198657d1 service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:02.977218 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92295b-fc5f-4f7b-8343-4caf198657d1 service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:02.977866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92295b-fc5f-4f7b-8343-4caf198657d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:03.046365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92295b-fc5f-4f7b-8343-4caf198657d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:03.047871 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92295b-fc5f-4f7b-8343-4caf198657d1 service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:03.058705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1377/2711] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:43:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:43:03.350653 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514ff69-fd62-4633-b960-1d62817848ff None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:03.350933 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514ff69-fd62-4633-b960-1d62817848ff None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:03.353728 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514ff69-fd62-4633-b960-1d62817848ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:03.376472 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7514ff69-fd62-4633-b960-1d62817848ff None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:43:03.377767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7514ff69-fd62-4633-b960-1d62817848ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:03.459638 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1335/2712] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:43:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:43:03.643000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92295b-fc5f-4f7b-8343-4caf198657d1 service glance] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:03.647389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4b92295b-fc5f-4f7b-8343-4caf198657d1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:03.811481 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92295b-fc5f-4f7b-8343-4caf198657d1 service glance] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:03.817479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a278a8-ead9-461c-9af1-0dce102a6cd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:03.821941 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a278a8-ead9-461c-9af1-0dce102a6cd2 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:03.822558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a278a8-ead9-461c-9af1-0dce102a6cd2 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:03.822859 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a278a8-ead9-461c-9af1-0dce102a6cd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:03.833297 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a278a8-ead9-461c-9af1-0dce102a6cd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:03.835016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a278a8-ead9-461c-9af1-0dce102a6cd2 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:03.845130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1378/2713] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:43:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:43:07.635775 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514ff69-fd62-4633-b960-1d62817848ff None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:07.800671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514ff69-fd62-4633-b960-1d62817848ff None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:07.806930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bc70c-8cad-468a-9f62-8443c3f858cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:07.808341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bc70c-8cad-468a-9f62-8443c3f858cb service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:07.815456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bc70c-8cad-468a-9f62-8443c3f858cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:07.815456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bc70c-8cad-468a-9f62-8443c3f858cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:07.815456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-986bc70c-8cad-468a-9f62-8443c3f858cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:07.834527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986bc70c-8cad-468a-9f62-8443c3f858cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:07.836976 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986bc70c-8cad-468a-9f62-8443c3f858cb service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:07.855275 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1336/2714] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:43:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:43:13.841304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:15.685398 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a1fa8-054c-4292-a3d0-758150b3d502 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:15.686042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a1fa8-054c-4292-a3d0-758150b3d502 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:15.686042 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a1fa8-054c-4292-a3d0-758150b3d502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:15.705742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2e0a1fa8-054c-4292-a3d0-758150b3d502 None None] MFA Rules not processed for user `196e0d38ea1d4669a3185cef7edef14a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:43:15.707194 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2e0a1fa8-054c-4292-a3d0-758150b3d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:15.714065 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1379/2715] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:43:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:43:17.847150 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=52 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:19.146477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a1658-ff03-4ffc-b12d-30d5ff8a04b0 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:19.147388 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a1658-ff03-4ffc-b12d-30d5ff8a04b0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:19.147713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a1658-ff03-4ffc-b12d-30d5ff8a04b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:19.168764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-769a1658-ff03-4ffc-b12d-30d5ff8a04b0 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:43:19.170258 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-769a1658-ff03-4ffc-b12d-30d5ff8a04b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:19.179042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1337/2716] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:43:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:43:19.191317 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a1fa8-054c-4292-a3d0-758150b3d502 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:19.203957 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a1fa8-054c-4292-a3d0-758150b3d502 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:19.209914 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b142e40-5efb-469e-a40d-f4337d90fe90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:19.210530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b142e40-5efb-469e-a40d-f4337d90fe90 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:19.210814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b142e40-5efb-469e-a40d-f4337d90fe90 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:19.211100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b142e40-5efb-469e-a40d-f4337d90fe90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:19.219474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b142e40-5efb-469e-a40d-f4337d90fe90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:19.221107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b142e40-5efb-469e-a40d-f4337d90fe90 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:19.229760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1380/2717] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:43:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:43:20.175317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a1658-ff03-4ffc-b12d-30d5ff8a04b0 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:20.197792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a1658-ff03-4ffc-b12d-30d5ff8a04b0 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:20.205929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295ef1-5e9f-49db-acbf-a949b8c32bc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:20.206574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295ef1-5e9f-49db-acbf-a949b8c32bc4 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:20.206853 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295ef1-5e9f-49db-acbf-a949b8c32bc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:20.207127 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295ef1-5e9f-49db-acbf-a949b8c32bc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:20.209322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8b295ef1-5e9f-49db-acbf-a949b8c32bc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:20.245529 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b295ef1-5e9f-49db-acbf-a949b8c32bc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:20.247164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b295ef1-5e9f-49db-acbf-a949b8c32bc4 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:20.276767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1338/2718] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:43:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:43:29.226464 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:30.263921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=9 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:31.295872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b142e40-5efb-469e-a40d-f4337d90fe90 service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:31.308368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b142e40-5efb-469e-a40d-f4337d90fe90 service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:31.313796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f628b-82b4-4b98-acce-2049fde35b7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:31.314387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f628b-82b4-4b98-acce-2049fde35b7d service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:31.314671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f628b-82b4-4b98-acce-2049fde35b7d service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:31.314942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f628b-82b4-4b98-acce-2049fde35b7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:31.316659 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e72f628b-82b4-4b98-acce-2049fde35b7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:31.327238 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f628b-82b4-4b98-acce-2049fde35b7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:31.328796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f628b-82b4-4b98-acce-2049fde35b7d service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:31.337771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1381/2719] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:43:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:43:37.741374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295ef1-5e9f-49db-acbf-a949b8c32bc4 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:37.756298 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295ef1-5e9f-49db-acbf-a949b8c32bc4 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:37.762310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ab2d1-e6ed-4ef9-b3a7-d55fe6ab94c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:37.762914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ab2d1-e6ed-4ef9-b3a7-d55fe6ab94c9 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:37.763189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ab2d1-e6ed-4ef9-b3a7-d55fe6ab94c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:37.763458 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ab2d1-e6ed-4ef9-b3a7-d55fe6ab94c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:37.765191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8c9ab2d1-e6ed-4ef9-b3a7-d55fe6ab94c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:37.778780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ab2d1-e6ed-4ef9-b3a7-d55fe6ab94c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:37.780377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ab2d1-e6ed-4ef9-b3a7-d55fe6ab94c9 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:37.792287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1339/2720] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:43:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:43:41.333660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:47.788492 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=6 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:53.431088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f628b-82b4-4b98-acce-2049fde35b7d service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:53.440382 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f628b-82b4-4b98-acce-2049fde35b7d service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:53.444910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013a809-6c4d-4ce2-b5c7-00d299843bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:53.445908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013a809-6c4d-4ce2-b5c7-00d299843bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:53.446288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013a809-6c4d-4ce2-b5c7-00d299843bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:53.446560 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013a809-6c4d-4ce2-b5c7-00d299843bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:53.455726 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013a809-6c4d-4ce2-b5c7-00d299843bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:53.457160 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013a809-6c4d-4ce2-b5c7-00d299843bb1 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:53.466528 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1382/2721] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:43:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:43:56.743753 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ab2d1-e6ed-4ef9-b3a7-d55fe6ab94c9 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:56.753144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ab2d1-e6ed-4ef9-b3a7-d55fe6ab94c9 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:56.756887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90754989-355c-4163-80a7-f3de1a6101c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:56.757490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90754989-355c-4163-80a7-f3de1a6101c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:56.757773 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90754989-355c-4163-80a7-f3de1a6101c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:56.758045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90754989-355c-4163-80a7-f3de1a6101c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:56.766851 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90754989-355c-4163-80a7-f3de1a6101c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:56.768264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90754989-355c-4163-80a7-f3de1a6101c5 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:56.777942 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1340/2722] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:43:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:44:03.462517 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:44:06.773897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=6 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:44:58.259405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013a809-6c4d-4ce2-b5c7-00d299843bb1 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:44:58.271855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013a809-6c4d-4ce2-b5c7-00d299843bb1 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:44:58.276769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c8506-2b8a-4a52-a53b-496bf3b86a1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:44:58.277571 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c8506-2b8a-4a52-a53b-496bf3b86a1a service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:44:58.278085 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c8506-2b8a-4a52-a53b-496bf3b86a1a service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:44:58.278565 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c8506-2b8a-4a52-a53b-496bf3b86a1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:44:58.281360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-847c8506-2b8a-4a52-a53b-496bf3b86a1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:44:58.297219 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c8506-2b8a-4a52-a53b-496bf3b86a1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:44:58.298877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c8506-2b8a-4a52-a53b-496bf3b86a1a service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:44:58.309939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1383/2723] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:44:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:45:08.304608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:45:19.642528 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8b268-f9a3-4378-95c7-c26c8bbaca9e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:19.643016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8b268-f9a3-4378-95c7-c26c8bbaca9e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:19.643121 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8b268-f9a3-4378-95c7-c26c8bbaca9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:19.662929 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-53b8b268-f9a3-4378-95c7-c26c8bbaca9e None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:45:19.664115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-53b8b268-f9a3-4378-95c7-c26c8bbaca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:19.671475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1341/2724] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:45:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:45:19.712449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c8506-2b8a-4a52-a53b-496bf3b86a1a service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:45:19.724405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c8506-2b8a-4a52-a53b-496bf3b86a1a service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:45:19.729509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11228c-ba55-46c2-acab-9fd867254613 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:45:19.730575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11228c-ba55-46c2-acab-9fd867254613 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:19.730883 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11228c-ba55-46c2-acab-9fd867254613 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:19.731155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11228c-ba55-46c2-acab-9fd867254613 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:19.739244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e11228c-ba55-46c2-acab-9fd867254613 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:45:19.740809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e11228c-ba55-46c2-acab-9fd867254613 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:45:19.749363 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1384/2725] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:45:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:45:29.657993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:45:29.745998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:45:36.418380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8b268-f9a3-4378-95c7-c26c8bbaca9e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:45:36.427841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8b268-f9a3-4378-95c7-c26c8bbaca9e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:45:36.431394 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e31a9-eb45-4ac6-b395-6d92d2689141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:45:36.431983 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e31a9-eb45-4ac6-b395-6d92d2689141 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:36.432256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e31a9-eb45-4ac6-b395-6d92d2689141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:36.432555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e31a9-eb45-4ac6-b395-6d92d2689141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:36.434264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3a3e31a9-eb45-4ac6-b395-6d92d2689141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:36.443658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e31a9-eb45-4ac6-b395-6d92d2689141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:45:36.445080 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e31a9-eb45-4ac6-b395-6d92d2689141 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:45:36.453972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1342/2726] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:45:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:45:39.688577 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a5dd1-d5f2-4234-b56c-6d635f33a1c1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:39.689027 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a5dd1-d5f2-4234-b56c-6d635f33a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:39.689138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a5dd1-d5f2-4234-b56c-6d635f33a1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:39.708402 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e31a5dd1-d5f2-4234-b56c-6d635f33a1c1 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:45:39.709714 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e31a5dd1-d5f2-4234-b56c-6d635f33a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:39.717510 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1385/2727] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:45:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:45:39.729595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e31a9-eb45-4ac6-b395-6d92d2689141 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:45:39.742225 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e31a9-eb45-4ac6-b395-6d92d2689141 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:45:39.747081 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ea986-f9dc-4e2d-a581-45b2e59a80c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:45:39.747675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea986-f9dc-4e2d-a581-45b2e59a80c5 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:39.747979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea986-f9dc-4e2d-a581-45b2e59a80c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:39.748248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea986-f9dc-4e2d-a581-45b2e59a80c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:39.756073 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ea986-f9dc-4e2d-a581-45b2e59a80c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:45:39.757670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ea986-f9dc-4e2d-a581-45b2e59a80c5 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:45:39.765654 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1343/2728] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:45:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:45:41.601985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a5dd1-d5f2-4234-b56c-6d635f33a1c1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:45:41.611783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a5dd1-d5f2-4234-b56c-6d635f33a1c1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:45:41.615654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf4eb2-a7c5-407c-8547-ce73b10d7de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:45:41.616326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf4eb2-a7c5-407c-8547-ce73b10d7de1 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:41.616671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf4eb2-a7c5-407c-8547-ce73b10d7de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:41.617007 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf4eb2-a7c5-407c-8547-ce73b10d7de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:41.618795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-72cf4eb2-a7c5-407c-8547-ce73b10d7de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:41.632735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf4eb2-a7c5-407c-8547-ce73b10d7de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:45:41.634201 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf4eb2-a7c5-407c-8547-ce73b10d7de1 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:45:41.645671 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1386/2729] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:45:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:45:42.866112 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ea986-f9dc-4e2d-a581-45b2e59a80c5 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:45:42.876625 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ea986-f9dc-4e2d-a581-45b2e59a80c5 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:45:42.880356 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4d510-0380-4218-abe9-05cedce7d01d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:45:42.880943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4d510-0380-4218-abe9-05cedce7d01d service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:42.881215 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4d510-0380-4218-abe9-05cedce7d01d service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:42.881510 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4d510-0380-4218-abe9-05cedce7d01d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:42.883212 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-eff4d510-0380-4218-abe9-05cedce7d01d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:42.896972 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4d510-0380-4218-abe9-05cedce7d01d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:45:42.898453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4d510-0380-4218-abe9-05cedce7d01d service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:45:42.910650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1344/2730] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:45:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:45:51.640752 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:45:52.905156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=18 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:46:00.352378 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b05915-a24b-4ccd-a069-5a9677b879e1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:00.352831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b05915-a24b-4ccd-a069-5a9677b879e1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:00.352831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b05915-a24b-4ccd-a069-5a9677b879e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:00.372485 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b8b05915-a24b-4ccd-a069-5a9677b879e1 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:46:00.373724 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b8b05915-a24b-4ccd-a069-5a9677b879e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:46:00.381315 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1387/2731] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 1 05:46:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:46:00.394228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4d510-0380-4218-abe9-05cedce7d01d service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:46:00.406621 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4d510-0380-4218-abe9-05cedce7d01d service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:46:00.412003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cbdad-206c-4e93-ae46-d9201e480215 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:46:00.412594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cbdad-206c-4e93-ae46-d9201e480215 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:00.412918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cbdad-206c-4e93-ae46-d9201e480215 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:00.413196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cbdad-206c-4e93-ae46-d9201e480215 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:00.421760 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cbdad-206c-4e93-ae46-d9201e480215 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:46:00.423336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cbdad-206c-4e93-ae46-d9201e480215 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:46:00.431778 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1345/2732] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:46:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:46:10.366986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:46:10.428456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=6 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:46:30.470538 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb8eb5-a3d5-41df-a69e-aadde2c21ece None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:30.473025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb8eb5-a3d5-41df-a69e-aadde2c21ece None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:30.473025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb8eb5-a3d5-41df-a69e-aadde2c21ece None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:30.473025 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1388/2733] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:46:30.478881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2335e3-1875-4567-968c-0b40a39a6d86 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:30.479418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2335e3-1875-4567-968c-0b40a39a6d86 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:30.479698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2335e3-1875-4567-968c-0b40a39a6d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:30.542183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8d2335e3-1875-4567-968c-0b40a39a6d86 None None] MFA Rules not processed for user `0e21910fca40479ca04e7f0418f7dd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:46:30.543401 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8d2335e3-1875-4567-968c-0b40a39a6d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:46:30.550341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1346/2734] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:46:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:46:30.563928 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb8eb5-a3d5-41df-a69e-aadde2c21ece None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:46:30.565785 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-16bb8eb5-a3d5-41df-a69e-aadde2c21ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:46:30.581319 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb8eb5-a3d5-41df-a69e-aadde2c21ece None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:46:30.586057 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51767c27-a501-462b-b322-82e5f5974cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:46:30.586664 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51767c27-a501-462b-b322-82e5f5974cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:30.587122 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51767c27-a501-462b-b322-82e5f5974cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:30.587559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51767c27-a501-462b-b322-82e5f5974cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:30.595544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51767c27-a501-462b-b322-82e5f5974cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:46:30.596972 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51767c27-a501-462b-b322-82e5f5974cd9 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:46:30.604989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1389/2735] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:46:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:46:30.689042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955be7e-4ad1-412b-8f29-143afb8efb12 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:30.689325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955be7e-4ad1-412b-8f29-143afb8efb12 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:30.689632 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955be7e-4ad1-412b-8f29-143afb8efb12 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:30.691115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1347/2736] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 1 05:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:46:30.696331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53f42d-ee1c-4b5e-bdcf-d4e9adae0a5f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:30.696618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53f42d-ee1c-4b5e-bdcf-d4e9adae0a5f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:30.696895 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53f42d-ee1c-4b5e-bdcf-d4e9adae0a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:30.721325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7e53f42d-ee1c-4b5e-bdcf-d4e9adae0a5f None None] MFA Rules not processed for user `7cda3c1f67e54ee08abc014399b07cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:46:30.722707 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7e53f42d-ee1c-4b5e-bdcf-d4e9adae0a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:46:30.733913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1390/2737] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 1 05:46:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:46:30.745089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955be7e-4ad1-412b-8f29-143afb8efb12 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:46:30.757768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955be7e-4ad1-412b-8f29-143afb8efb12 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:46:30.761320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0e1f7-1fac-454b-89ef-ab81b9b729d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:46:30.761985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0e1f7-1fac-454b-89ef-ab81b9b729d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:30.762259 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0e1f7-1fac-454b-89ef-ab81b9b729d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:30.762527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0e1f7-1fac-454b-89ef-ab81b9b729d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:30.774070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0e1f7-1fac-454b-89ef-ab81b9b729d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:46:30.775492 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0e1f7-1fac-454b-89ef-ab81b9b729d1 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:46:30.786948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1348/2738] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:46:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:46:35.882135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169d77a-479d-421f-9590-c30deb50a5f3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:35.882685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169d77a-479d-421f-9590-c30deb50a5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:35.882685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169d77a-479d-421f-9590-c30deb50a5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:35.901627 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f169d77a-479d-421f-9590-c30deb50a5f3 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:46:35.902815 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f169d77a-479d-421f-9590-c30deb50a5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:46:35.911202 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1391/2739] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 1 05:46:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:46:35.932974 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0e1f7-1fac-454b-89ef-ab81b9b729d1 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:46:35.944592 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0e1f7-1fac-454b-89ef-ab81b9b729d1 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:46:35.949802 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e97c62-179c-4524-825d-27f876cd4901 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:46:35.950815 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e97c62-179c-4524-825d-27f876cd4901 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:35.951095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e97c62-179c-4524-825d-27f876cd4901 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:35.951363 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e97c62-179c-4524-825d-27f876cd4901 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:35.959332 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e97c62-179c-4524-825d-27f876cd4901 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:46:35.960902 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e97c62-179c-4524-825d-27f876cd4901 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:46:35.969357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1349/2740] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:46:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:46:44.387237 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169d77a-479d-421f-9590-c30deb50a5f3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:46:44.474698 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169d77a-479d-421f-9590-c30deb50a5f3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:46:44.479831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:46:44.480494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:44.480826 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:44.481102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] PATH_INFO: `/v3/users/196e0d38ea1d4669a3185cef7edef14a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:44.482623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=196e0d38ea1d4669a3185cef7edef14a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:46:44.484392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:46:44.668183 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196e0d38ea1d4669a3185cef7edef14a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:44.681530 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196e0d38ea1d4669a3185cef7edef14a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:44.689515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196e0d38ea1d4669a3185cef7edef14a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:44.757075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196e0d38ea1d4669a3185cef7edef14a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:44.757739 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196e0d38ea1d4669a3185cef7edef14a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:44.765092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1392/2741] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:46:44 2022] DELETE /identity/v3/users/196e0d38ea1d4669a3185cef7edef14a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:46:45.044434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e97c62-179c-4524-825d-27f876cd4901 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:46:45.057767 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e97c62-179c-4524-825d-27f876cd4901 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:46:45.119152 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:46:45.119952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:45.120363 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:45.120643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] PATH_INFO: `/v3/projects/1f39d8d2806b485984009aab93cb6f9d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:45.121797 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f39d8d2806b485984009aab93cb6f9d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:46:45.123900 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:46:45.204815 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f39d8d2806b485984009aab93cb6f9d', 'request_id': 'req-d5a38922-9aa5-43c0-8b1f-153fca13885c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:45.259920 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f39d8d2806b485984009aab93cb6f9d', 'request_id': 'req-d5a38922-9aa5-43c0-8b1f-153fca13885c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:45.267042 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f39d8d2806b485984009aab93cb6f9d', 'request_id': 'req-d5a38922-9aa5-43c0-8b1f-153fca13885c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:45.277871 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] The token cache is being invalidate because project 1f39d8d2806b485984009aab93cb6f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:46:45.278156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:45.278800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:45.280527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1350/2742] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:46:45 2022] DELETE /identity/v3/projects/1f39d8d2806b485984009aab93cb6f9d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:46:45.284699 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:46:45.286195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:46:45.299873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9533a6-89c8-4983-90c3-5a3feb3d3d7c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:46:45.349431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:46:45.358719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:46:45.359344 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:45.359887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:45.359887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] PATH_INFO: `/v3/users/07d81435cf3c4cebb1caaec654586638` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:45.361116 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d81435cf3c4cebb1caaec654586638)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:46:45.363044 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:46:45.564424 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d81435cf3c4cebb1caaec654586638'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:45.576647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d81435cf3c4cebb1caaec654586638'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:45.583413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d81435cf3c4cebb1caaec654586638'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:45.676965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d81435cf3c4cebb1caaec654586638'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:45.677624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d81435cf3c4cebb1caaec654586638'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:45.684089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1393/2743] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:46:45 2022] DELETE /identity/v3/users/07d81435cf3c4cebb1caaec654586638 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:46:46.154764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:46:46.155503 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:46:46.168430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a38922-9aa5-43c0-8b1f-153fca13885c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:46:46.218543 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:46:46.234047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:46:46.236660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:46:46.236660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:46:46.236660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] PATH_INFO: `/v3/projects/9ba17fb8e83e4648b0428141a5755f05` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:46:46.236660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ba17fb8e83e4648b0428141a5755f05)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:46:46.238231 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:46:46.336746 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ba17fb8e83e4648b0428141a5755f05', 'request_id': 'req-c72115db-537a-4e4a-8e12-7a66ad037d7e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:46.414014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ba17fb8e83e4648b0428141a5755f05', 'request_id': 'req-c72115db-537a-4e4a-8e12-7a66ad037d7e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:46.422229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ba17fb8e83e4648b0428141a5755f05', 'request_id': 'req-c72115db-537a-4e4a-8e12-7a66ad037d7e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:46.433418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] The token cache is being invalidate because project 9ba17fb8e83e4648b0428141a5755f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:46:46.434572 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:46.435265 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:46:46.436784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1351/2744] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:46:46 2022] DELETE /identity/v3/projects/9ba17fb8e83e4648b0428141a5755f05 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:46:55.680365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:46:56.430338 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:07.355617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:07.357048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:07.415428 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79be5f5-6f5f-46a9-9887-928a67e90dc5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:07.463273 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6260b-ae7c-4765-8c8c-667094ce4240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:07.463889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6260b-ae7c-4765-8c8c-667094ce4240 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:07.464172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6260b-ae7c-4765-8c8c-667094ce4240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:07.464444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6260b-ae7c-4765-8c8c-667094ce4240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:07.466230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e9b6260b-ae7c-4765-8c8c-667094ce4240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:07.529430 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6260b-ae7c-4765-8c8c-667094ce4240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:07.531923 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6260b-ae7c-4765-8c8c-667094ce4240 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:07.541414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1394/2745] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:11.115768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:11.118237 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:11.192323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72115db-537a-4e4a-8e12-7a66ad037d7e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:11.197675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3257c-a35c-4088-9f10-b6c40b9d1df1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:11.198271 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3257c-a35c-4088-9f10-b6c40b9d1df1 service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:11.198717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3257c-a35c-4088-9f10-b6c40b9d1df1 service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:11.198986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3257c-a35c-4088-9f10-b6c40b9d1df1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:11.200854 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-07d3257c-a35c-4088-9f10-b6c40b9d1df1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:11.212982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3257c-a35c-4088-9f10-b6c40b9d1df1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:11.214624 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3257c-a35c-4088-9f10-b6c40b9d1df1 service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:11.225641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1352/2746] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:11.232132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6260b-ae7c-4765-8c8c-667094ce4240 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:11.240199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e9b6260b-ae7c-4765-8c8c-667094ce4240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:11.250865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6260b-ae7c-4765-8c8c-667094ce4240 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:11.258623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:11.258623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:11.258623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:11.258623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] PATH_INFO: `/v3/users/1aa49a620d0e42b0947215d200fa45a1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:11.259110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aa49a620d0e42b0947215d200fa45a1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:11.260167 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:11.409734 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aa49a620d0e42b0947215d200fa45a1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:11.419732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:11.422613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aa49a620d0e42b0947215d200fa45a1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:11.429285 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aa49a620d0e42b0947215d200fa45a1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:11.476392 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aa49a620d0e42b0947215d200fa45a1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:11.477016 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-32fc91c3-8b36-464f-97a2-1f93aaa7501f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aa49a620d0e42b0947215d200fa45a1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:11.483751 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1395/2747] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:47:11 2022] DELETE /identity/v3/users/1aa49a620d0e42b0947215d200fa45a1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:11.729810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3257c-a35c-4088-9f10-b6c40b9d1df1 service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:11.745325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3257c-a35c-4088-9f10-b6c40b9d1df1 service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:11.815553 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:11.816193 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:11.816473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:11.816752 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] PATH_INFO: `/v3/projects/459dd5f92ff94f8b81d1e7213e8e7170` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:11.818012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=459dd5f92ff94f8b81d1e7213e8e7170)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:11.819945 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:11.917248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459dd5f92ff94f8b81d1e7213e8e7170', 'request_id': 'req-5cf98020-3c00-493e-925f-ad6ae90990bd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:11.973376 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459dd5f92ff94f8b81d1e7213e8e7170', 'request_id': 'req-5cf98020-3c00-493e-925f-ad6ae90990bd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:11.980164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459dd5f92ff94f8b81d1e7213e8e7170', 'request_id': 'req-5cf98020-3c00-493e-925f-ad6ae90990bd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:11.990852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] The token cache is being invalidate because project 459dd5f92ff94f8b81d1e7213e8e7170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:47:11.991142 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:11.991796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cf98020-3c00-493e-925f-ad6ae90990bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:11.993285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1353/2748] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:47:11 2022] DELETE /identity/v3/projects/459dd5f92ff94f8b81d1e7213e8e7170 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:11.998206 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d44d78-c966-48cd-8bdc-ed70cc9378b5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:11.998496 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d44d78-c966-48cd-8bdc-ed70cc9378b5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:11.998835 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d44d78-c966-48cd-8bdc-ed70cc9378b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.019068 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c0d44d78-c966-48cd-8bdc-ed70cc9378b5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:12.020276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c0d44d78-c966-48cd-8bdc-ed70cc9378b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:12.080619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1396/2749] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:47:12.088682 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd148e-568d-4500-9f32-77f71b2ebf13 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:12.088970 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd148e-568d-4500-9f32-77f71b2ebf13 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:12.089246 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd148e-568d-4500-9f32-77f71b2ebf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.109876 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-81fd148e-568d-4500-9f32-77f71b2ebf13 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:12.111088 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-81fd148e-568d-4500-9f32-77f71b2ebf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:12.118962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1354/2750] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:47:12.123200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d44d78-c966-48cd-8bdc-ed70cc9378b5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:12.132743 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d44d78-c966-48cd-8bdc-ed70cc9378b5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:12.137368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4351a-5660-4a7f-8561-0bc2b61557c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:12.138002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4351a-5660-4a7f-8561-0bc2b61557c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:12.138281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4351a-5660-4a7f-8561-0bc2b61557c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:12.138855 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4351a-5660-4a7f-8561-0bc2b61557c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.139199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4351a-5660-4a7f-8561-0bc2b61557c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:47:12.140097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4351a-5660-4a7f-8561-0bc2b61557c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:12.141910 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4351a-5660-4a7f-8561-0bc2b61557c4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:12.153715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1397/2751] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:47:12.158658 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd148e-568d-4500-9f32-77f71b2ebf13 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:12.168293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd148e-568d-4500-9f32-77f71b2ebf13 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:12.173360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8cbf1-c7ba-4652-b2ed-f44f511d79c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:12.173992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8cbf1-c7ba-4652-b2ed-f44f511d79c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:12.179230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8cbf1-c7ba-4652-b2ed-f44f511d79c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:12.179230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8cbf1-c7ba-4652-b2ed-f44f511d79c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.179230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8cbf1-c7ba-4652-b2ed-f44f511d79c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:12.179230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8cbf1-c7ba-4652-b2ed-f44f511d79c2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:12.188372 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:12.260554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1355/2752] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:47:12 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:47:12.265553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4351a-5660-4a7f-8561-0bc2b61557c4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:12.275976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4351a-5660-4a7f-8561-0bc2b61557c4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:12.334047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed6a90-719c-4332-a7b2-8399183af950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:12.334685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed6a90-719c-4332-a7b2-8399183af950 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:12.334967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed6a90-719c-4332-a7b2-8399183af950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:12.335239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed6a90-719c-4332-a7b2-8399183af950 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.335894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed6a90-719c-4332-a7b2-8399183af950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:12.337887 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed6a90-719c-4332-a7b2-8399183af950 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:12.419491 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1398/2753] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:47:12 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:47:12.423884 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8cbf1-c7ba-4652-b2ed-f44f511d79c2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:12.434716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8cbf1-c7ba-4652-b2ed-f44f511d79c2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:12.439863 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00c2a9-0e49-4b6f-b78d-ceb0b87633b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:12.440490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00c2a9-0e49-4b6f-b78d-ceb0b87633b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:12.440838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00c2a9-0e49-4b6f-b78d-ceb0b87633b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:12.441111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00c2a9-0e49-4b6f-b78d-ceb0b87633b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.441862 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00c2a9-0e49-4b6f-b78d-ceb0b87633b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:12.443450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00c2a9-0e49-4b6f-b78d-ceb0b87633b6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:12.454407 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1356/2754] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:47:12.458487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed6a90-719c-4332-a7b2-8399183af950 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:12.469140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed6a90-719c-4332-a7b2-8399183af950 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:12.474037 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648c631-8bc4-4dc9-a3af-5fd98e917f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:12.474659 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648c631-8bc4-4dc9-a3af-5fd98e917f24 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:12.474935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648c631-8bc4-4dc9-a3af-5fd98e917f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:12.475205 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648c631-8bc4-4dc9-a3af-5fd98e917f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.475842 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648c631-8bc4-4dc9-a3af-5fd98e917f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:12.477449 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648c631-8bc4-4dc9-a3af-5fd98e917f24 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:12.488035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1399/2755] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:47:12.492063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00c2a9-0e49-4b6f-b78d-ceb0b87633b6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:12.502618 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00c2a9-0e49-4b6f-b78d-ceb0b87633b6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:12.507637 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00aa793-a2a1-4d8a-894b-c21005ce7290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:12.508281 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00aa793-a2a1-4d8a-894b-c21005ce7290 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:12.508559 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00aa793-a2a1-4d8a-894b-c21005ce7290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:12.508830 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00aa793-a2a1-4d8a-894b-c21005ce7290 admin admin] PATH_INFO: `/v3/projects/5716606469894079ab49e9cb173a49a0/users/76a38407f7964873bc853cd679546e25/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.524765 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00aa793-a2a1-4d8a-894b-c21005ce7290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5716606469894079ab49e9cb173a49a0, user_id=76a38407f7964873bc853cd679546e25, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:12.527523 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00aa793-a2a1-4d8a-894b-c21005ce7290 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:12.580381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1357/2756] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:47:12 2022] PUT /identity/v3/projects/5716606469894079ab49e9cb173a49a0/users/76a38407f7964873bc853cd679546e25/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:12.591725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a829a38-65d4-4cd0-b255-e1237b480e95 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:12.592062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a829a38-65d4-4cd0-b255-e1237b480e95 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:12.592359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a829a38-65d4-4cd0-b255-e1237b480e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.625427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0a829a38-65d4-4cd0-b255-e1237b480e95 None None] MFA Rules not processed for user `76a38407f7964873bc853cd679546e25`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:12.626979 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0a829a38-65d4-4cd0-b255-e1237b480e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:12.664765 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:12.701660 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1400/2757] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:47:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:47:12.705032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00aa793-a2a1-4d8a-894b-c21005ce7290 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:12.768400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00aa793-a2a1-4d8a-894b-c21005ce7290 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:12.773148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740ad00-94b4-4f7c-af9d-ba4954e33607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:12.773789 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740ad00-94b4-4f7c-af9d-ba4954e33607 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:12.774130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740ad00-94b4-4f7c-af9d-ba4954e33607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:12.774411 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740ad00-94b4-4f7c-af9d-ba4954e33607 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.775103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740ad00-94b4-4f7c-af9d-ba4954e33607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:12.776962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740ad00-94b4-4f7c-af9d-ba4954e33607 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:12.861906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1358/2758] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:47:12 2022] POST /identity/v3/projects => generated 369 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:47:12.866241 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a829a38-65d4-4cd0-b255-e1237b480e95 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:12.877101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a829a38-65d4-4cd0-b255-e1237b480e95 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:12.934956 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752835d-b1a1-4dff-9ce2-dad5c0157d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:12.935564 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752835d-b1a1-4dff-9ce2-dad5c0157d25 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:12.935848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752835d-b1a1-4dff-9ce2-dad5c0157d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:12.936120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752835d-b1a1-4dff-9ce2-dad5c0157d25 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:12.936746 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752835d-b1a1-4dff-9ce2-dad5c0157d25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:12.938857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752835d-b1a1-4dff-9ce2-dad5c0157d25 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:13.048723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1401/2759] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:47:12 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:47:13.052641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740ad00-94b4-4f7c-af9d-ba4954e33607 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:13.063903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740ad00-94b4-4f7c-af9d-ba4954e33607 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:13.068462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23301517-a6b9-4e6c-b1d2-7f8a9b1c3c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:13.069053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23301517-a6b9-4e6c-b1d2-7f8a9b1c3c25 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:13.069342 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23301517-a6b9-4e6c-b1d2-7f8a9b1c3c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:13.069654 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23301517-a6b9-4e6c-b1d2-7f8a9b1c3c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:13.070288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23301517-a6b9-4e6c-b1d2-7f8a9b1c3c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:13.072125 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23301517-a6b9-4e6c-b1d2-7f8a9b1c3c25 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:13.083188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1359/2760] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:47:13.087275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752835d-b1a1-4dff-9ce2-dad5c0157d25 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:13.097824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752835d-b1a1-4dff-9ce2-dad5c0157d25 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:13.102440 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44584384-91fe-4c03-81e1-1c8738e62bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:13.103084 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44584384-91fe-4c03-81e1-1c8738e62bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:13.103361 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44584384-91fe-4c03-81e1-1c8738e62bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:13.103634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44584384-91fe-4c03-81e1-1c8738e62bf9 admin admin] PATH_INFO: `/v3/projects/886a4c4bdef443959a161b5f0417e56d/users/26a497eb721b44c792d73f5ad51b235a/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:13.118784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44584384-91fe-4c03-81e1-1c8738e62bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=886a4c4bdef443959a161b5f0417e56d, user_id=26a497eb721b44c792d73f5ad51b235a, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:13.121420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44584384-91fe-4c03-81e1-1c8738e62bf9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:13.219474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1402/2761] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:47:13 2022] PUT /identity/v3/projects/886a4c4bdef443959a161b5f0417e56d/users/26a497eb721b44c792d73f5ad51b235a/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:13.223436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23301517-a6b9-4e6c-b1d2-7f8a9b1c3c25 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:13.234110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23301517-a6b9-4e6c-b1d2-7f8a9b1c3c25 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:13.293062 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dfe8b-1016-460e-8535-a2240a018b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:13.293708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dfe8b-1016-460e-8535-a2240a018b74 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:13.293992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dfe8b-1016-460e-8535-a2240a018b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:13.294274 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dfe8b-1016-460e-8535-a2240a018b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:13.294908 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57dfe8b-1016-460e-8535-a2240a018b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:13.296507 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57dfe8b-1016-460e-8535-a2240a018b74 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:13.306597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1360/2762] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:47:13.311204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44584384-91fe-4c03-81e1-1c8738e62bf9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:13.326456 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44584384-91fe-4c03-81e1-1c8738e62bf9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:13.333805 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a9f43-ed90-46ae-92fa-3f5ba4ded8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:13.334479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a9f43-ed90-46ae-92fa-3f5ba4ded8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:13.334885 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a9f43-ed90-46ae-92fa-3f5ba4ded8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:13.335365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a9f43-ed90-46ae-92fa-3f5ba4ded8bf admin admin] PATH_INFO: `/v3/projects/886a4c4bdef443959a161b5f0417e56d/users/26a497eb721b44c792d73f5ad51b235a/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:13.338688 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a9f43-ed90-46ae-92fa-3f5ba4ded8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=886a4c4bdef443959a161b5f0417e56d, user_id=26a497eb721b44c792d73f5ad51b235a, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:13.343379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a9f43-ed90-46ae-92fa-3f5ba4ded8bf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:13.355753 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1403/2763] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:47:13 2022] PUT /identity/v3/projects/886a4c4bdef443959a161b5f0417e56d/users/26a497eb721b44c792d73f5ad51b235a/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:13.368603 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dacdc0-e865-41a5-a146-3e6349bf42de None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:13.368890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dacdc0-e865-41a5-a146-3e6349bf42de None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:13.369167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dacdc0-e865-41a5-a146-3e6349bf42de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:13.407239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-95dacdc0-e865-41a5-a146-3e6349bf42de None None] MFA Rules not processed for user `26a497eb721b44c792d73f5ad51b235a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:13.408487 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-95dacdc0-e865-41a5-a146-3e6349bf42de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:13.422796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:13.478875 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1361/2764] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:47:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:47:13.490824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a9f43-ed90-46ae-92fa-3f5ba4ded8bf admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:13.492622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9d3a9f43-ed90-46ae-92fa-3f5ba4ded8bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:13.557632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a9f43-ed90-46ae-92fa-3f5ba4ded8bf admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:13.561159 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730cad9-94bb-44fc-85c4-d0d5489b69cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:13.561969 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730cad9-94bb-44fc-85c4-d0d5489b69cb service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:13.562438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730cad9-94bb-44fc-85c4-d0d5489b69cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:13.563486 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730cad9-94bb-44fc-85c4-d0d5489b69cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:13.573865 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730cad9-94bb-44fc-85c4-d0d5489b69cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:13.575444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730cad9-94bb-44fc-85c4-d0d5489b69cb service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:13.588292 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1404/2765] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:16.334477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dacdc0-e865-41a5-a146-3e6349bf42de None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:16.403392 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dacdc0-e865-41a5-a146-3e6349bf42de None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:16.409264 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:16.410003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:16.410314 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:16.410636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] PATH_INFO: `/v3/users/76a38407f7964873bc853cd679546e25` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:16.411949 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76a38407f7964873bc853cd679546e25)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:16.414232 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:16.634304 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76a38407f7964873bc853cd679546e25'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:16.634948 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76a38407f7964873bc853cd679546e25'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:16.711141 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76a38407f7964873bc853cd679546e25'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:16.726391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76a38407f7964873bc853cd679546e25'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:16.733744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76a38407f7964873bc853cd679546e25'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:16.746556 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1362/2766] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:47:16 2022] DELETE /identity/v3/users/76a38407f7964873bc853cd679546e25 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:16.754230 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a6b54-b225-40a7-b957-d40ce373edc1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:16.754789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a6b54-b225-40a7-b957-d40ce373edc1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:16.754894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a6b54-b225-40a7-b957-d40ce373edc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:16.780026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a29a6b54-b225-40a7-b957-d40ce373edc1 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:16.781886 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a29a6b54-b225-40a7-b957-d40ce373edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:16.806088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:16.841790 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1405/2767] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:47:16.852144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:16.854554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:16.888882 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:16.913601 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed31b3-cae4-4419-9a79-073ecc991458 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:16.917813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0261af6-94a7-469c-9c40-6af7096e4e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:16.918418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261af6-94a7-469c-9c40-6af7096e4e4c service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:16.918695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261af6-94a7-469c-9c40-6af7096e4e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:16.918965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261af6-94a7-469c-9c40-6af7096e4e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:16.928414 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0261af6-94a7-469c-9c40-6af7096e4e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:16.929953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0261af6-94a7-469c-9c40-6af7096e4e4c service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:16.940519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1363/2768] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:17.352472 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a6b54-b225-40a7-b957-d40ce373edc1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:17.353935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a29a6b54-b225-40a7-b957-d40ce373edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:17.366379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a6b54-b225-40a7-b957-d40ce373edc1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:17.371172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:17.371811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:17.372091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:17.372373 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] PATH_INFO: `/v3/projects/5716606469894079ab49e9cb173a49a0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:17.373567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5716606469894079ab49e9cb173a49a0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:17.376079 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:17.498138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5716606469894079ab49e9cb173a49a0', 'request_id': 'req-8131a790-cb4a-418c-bcd6-9e715fa92ee8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:17.590443 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5716606469894079ab49e9cb173a49a0', 'request_id': 'req-8131a790-cb4a-418c-bcd6-9e715fa92ee8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:17.597833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5716606469894079ab49e9cb173a49a0', 'request_id': 'req-8131a790-cb4a-418c-bcd6-9e715fa92ee8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:17.608000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] The token cache is being invalidate because project 5716606469894079ab49e9cb173a49a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:47:17.608288 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:17.608966 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:17.611035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1406/2769] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:47:17 2022] DELETE /identity/v3/projects/5716606469894079ab49e9cb173a49a0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:17.615003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0261af6-94a7-469c-9c40-6af7096e4e4c service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:17.624950 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0261af6-94a7-469c-9c40-6af7096e4e4c service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:17.680628 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:17.681298 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:17.681615 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:17.681890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] PATH_INFO: `/v3/users/26a497eb721b44c792d73f5ad51b235a` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:17.683000 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a497eb721b44c792d73f5ad51b235a)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:17.684934 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:17.850687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a497eb721b44c792d73f5ad51b235a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:17.851260 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a497eb721b44c792d73f5ad51b235a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:17.910322 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a497eb721b44c792d73f5ad51b235a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:17.922784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a497eb721b44c792d73f5ad51b235a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:17.929910 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-beb6f21b-4dd3-462e-8023-d8578c954c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a497eb721b44c792d73f5ad51b235a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:17.935745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1364/2770] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:47:17 2022] DELETE /identity/v3/users/26a497eb721b44c792d73f5ad51b235a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:18.336090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:18.337589 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:18.352520 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131a790-cb4a-418c-bcd6-9e715fa92ee8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:18.413298 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:18.414058 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:18.414349 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:18.414625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] PATH_INFO: `/v3/projects/886a4c4bdef443959a161b5f0417e56d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:18.415760 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=886a4c4bdef443959a161b5f0417e56d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:18.417722 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:18.542171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '886a4c4bdef443959a161b5f0417e56d', 'request_id': 'req-1f0a5036-4849-483a-98ff-6a3dc76d6513'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:18.546069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:18.601781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '886a4c4bdef443959a161b5f0417e56d', 'request_id': 'req-1f0a5036-4849-483a-98ff-6a3dc76d6513'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:18.608623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '886a4c4bdef443959a161b5f0417e56d', 'request_id': 'req-1f0a5036-4849-483a-98ff-6a3dc76d6513'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:18.621026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] The token cache is being invalidate because project 886a4c4bdef443959a161b5f0417e56d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:47:18.621245 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:18.621940 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f0a5036-4849-483a-98ff-6a3dc76d6513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:18.623482 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1407/2771] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:47:18 2022] DELETE /identity/v3/projects/886a4c4bdef443959a161b5f0417e56d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:18.629186 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142ac61-7785-4d5f-bc81-616eb6e65302 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:18.629708 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142ac61-7785-4d5f-bc81-616eb6e65302 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:18.629799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142ac61-7785-4d5f-bc81-616eb6e65302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:18.650252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9142ac61-7785-4d5f-bc81-616eb6e65302 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:18.651456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9142ac61-7785-4d5f-bc81-616eb6e65302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:18.699381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:18.712068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1365/2772] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:47:18.720515 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bebc38-995a-4b64-b4f5-d98c4c88f2cc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:18.720796 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bebc38-995a-4b64-b4f5-d98c4c88f2cc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:18.721071 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bebc38-995a-4b64-b4f5-d98c4c88f2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:18.741771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-25bebc38-995a-4b64-b4f5-d98c4c88f2cc None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:18.743689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-25bebc38-995a-4b64-b4f5-d98c4c88f2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:18.755614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1408/2773] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:47:18.759957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142ac61-7785-4d5f-bc81-616eb6e65302 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:18.769996 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142ac61-7785-4d5f-bc81-616eb6e65302 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:18.775160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5518860-a652-480b-b48f-30f199f0bd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:18.775756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518860-a652-480b-b48f-30f199f0bd55 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:18.776032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518860-a652-480b-b48f-30f199f0bd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:18.776305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518860-a652-480b-b48f-30f199f0bd55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:18.776981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5518860-a652-480b-b48f-30f199f0bd55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:47:18.777285 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5518860-a652-480b-b48f-30f199f0bd55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:18.779024 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5518860-a652-480b-b48f-30f199f0bd55 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:18.791367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1366/2774] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:47:18.795705 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bebc38-995a-4b64-b4f5-d98c4c88f2cc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:18.805679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bebc38-995a-4b64-b4f5-d98c4c88f2cc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:18.810404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9452893-ddf6-4f1f-82f2-ae71ff57b5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:18.811010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9452893-ddf6-4f1f-82f2-ae71ff57b5ae admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:18.811287 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9452893-ddf6-4f1f-82f2-ae71ff57b5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:18.811561 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9452893-ddf6-4f1f-82f2-ae71ff57b5ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:18.812191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9452893-ddf6-4f1f-82f2-ae71ff57b5ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:18.815041 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9452893-ddf6-4f1f-82f2-ae71ff57b5ae admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:18.884147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1409/2775] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:47:18 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:47:18.888377 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5518860-a652-480b-b48f-30f199f0bd55 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:18.898748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5518860-a652-480b-b48f-30f199f0bd55 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:18.957784 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54815f50-bde3-4442-b16b-6698a09e8b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:18.958379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54815f50-bde3-4442-b16b-6698a09e8b0f admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:18.958652 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54815f50-bde3-4442-b16b-6698a09e8b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:18.958919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54815f50-bde3-4442-b16b-6698a09e8b0f admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:18.959530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54815f50-bde3-4442-b16b-6698a09e8b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:18.961480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54815f50-bde3-4442-b16b-6698a09e8b0f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:19.030082 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1367/2776] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:47:18 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:47:19.034880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9452893-ddf6-4f1f-82f2-ae71ff57b5ae admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:19.045060 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9452893-ddf6-4f1f-82f2-ae71ff57b5ae admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:19.050221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16d26a-1c27-4f74-bf70-53317c892a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:19.050817 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16d26a-1c27-4f74-bf70-53317c892a4a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:19.051092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16d26a-1c27-4f74-bf70-53317c892a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:19.051360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16d26a-1c27-4f74-bf70-53317c892a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:19.052022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16d26a-1c27-4f74-bf70-53317c892a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:19.053630 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16d26a-1c27-4f74-bf70-53317c892a4a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:19.064221 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1410/2777] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:47:19.068146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54815f50-bde3-4442-b16b-6698a09e8b0f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:19.079021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54815f50-bde3-4442-b16b-6698a09e8b0f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:19.083598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1935-f75b-4782-9f9c-681324a8aa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:19.084175 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1935-f75b-4782-9f9c-681324a8aa7a admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:19.084449 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1935-f75b-4782-9f9c-681324a8aa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:19.084716 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1935-f75b-4782-9f9c-681324a8aa7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:19.085346 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a1935-f75b-4782-9f9c-681324a8aa7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:19.086957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a1935-f75b-4782-9f9c-681324a8aa7a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:19.098166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1368/2778] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:47:19.105946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16d26a-1c27-4f74-bf70-53317c892a4a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:19.116639 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16d26a-1c27-4f74-bf70-53317c892a4a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:19.121189 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5683cd-2fe4-4e3f-96dd-978fe556087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:19.121867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5683cd-2fe4-4e3f-96dd-978fe556087c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:19.122145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5683cd-2fe4-4e3f-96dd-978fe556087c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:19.122416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5683cd-2fe4-4e3f-96dd-978fe556087c admin admin] PATH_INFO: `/v3/projects/caa4ca3167f64aec83e75881392ba739/users/ca49a389c7b54f4c9ec5160a95bad4c3/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:19.138544 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5683cd-2fe4-4e3f-96dd-978fe556087c admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa4ca3167f64aec83e75881392ba739, user_id=ca49a389c7b54f4c9ec5160a95bad4c3, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:19.141685 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5683cd-2fe4-4e3f-96dd-978fe556087c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:19.202679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1411/2779] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:47:19 2022] PUT /identity/v3/projects/caa4ca3167f64aec83e75881392ba739/users/ca49a389c7b54f4c9ec5160a95bad4c3/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:47:19.213012 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbdea2-e4bb-491b-95bf-f9866177997a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:19.213297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbdea2-e4bb-491b-95bf-f9866177997a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:19.213610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbdea2-e4bb-491b-95bf-f9866177997a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:19.242473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1efbdea2-e4bb-491b-95bf-f9866177997a None None] MFA Rules not processed for user `ca49a389c7b54f4c9ec5160a95bad4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:19.243690 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1efbdea2-e4bb-491b-95bf-f9866177997a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:19.308218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1369/2780] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:47:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:47:19.312872 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5683cd-2fe4-4e3f-96dd-978fe556087c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:19.374036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5683cd-2fe4-4e3f-96dd-978fe556087c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:19.379683 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ba095-78fd-44fd-b38b-860f98a06a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:19.380271 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ba095-78fd-44fd-b38b-860f98a06a87 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:19.380552 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ba095-78fd-44fd-b38b-860f98a06a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:19.380822 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ba095-78fd-44fd-b38b-860f98a06a87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:19.381441 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ba095-78fd-44fd-b38b-860f98a06a87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:19.383329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ba095-78fd-44fd-b38b-860f98a06a87 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:19.478379 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1412/2781] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:47:19 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:47:19.482558 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbdea2-e4bb-491b-95bf-f9866177997a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:19.494240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbdea2-e4bb-491b-95bf-f9866177997a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:19.542155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:19.554458 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156282c7-75a8-413f-96fb-466029a0f45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:19.555056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156282c7-75a8-413f-96fb-466029a0f45c admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:19.555326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156282c7-75a8-413f-96fb-466029a0f45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:19.555595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156282c7-75a8-413f-96fb-466029a0f45c admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:19.556233 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156282c7-75a8-413f-96fb-466029a0f45c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:19.558156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156282c7-75a8-413f-96fb-466029a0f45c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:19.670426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1370/2782] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:47:19 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:47:19.674942 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ba095-78fd-44fd-b38b-860f98a06a87 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:19.685317 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ba095-78fd-44fd-b38b-860f98a06a87 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:19.690634 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78677924-a8b1-4225-8ae8-faf612331546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:19.691229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78677924-a8b1-4225-8ae8-faf612331546 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:19.691553 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78677924-a8b1-4225-8ae8-faf612331546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:19.691820 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78677924-a8b1-4225-8ae8-faf612331546 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:19.692462 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78677924-a8b1-4225-8ae8-faf612331546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:19.694123 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78677924-a8b1-4225-8ae8-faf612331546 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:19.704846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1413/2783] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:47:19.708906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156282c7-75a8-413f-96fb-466029a0f45c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:19.720261 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156282c7-75a8-413f-96fb-466029a0f45c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:19.725256 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a139853-8cfd-4331-81c8-e120a6a29400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:19.726057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a139853-8cfd-4331-81c8-e120a6a29400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:19.726400 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a139853-8cfd-4331-81c8-e120a6a29400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:19.726800 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a139853-8cfd-4331-81c8-e120a6a29400 admin admin] PATH_INFO: `/v3/projects/99635c9a2a00410c9df13fa6231c94f9/users/222fd54c252e4f1686690fb1a29d1fbc/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:19.743004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a139853-8cfd-4331-81c8-e120a6a29400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99635c9a2a00410c9df13fa6231c94f9, user_id=222fd54c252e4f1686690fb1a29d1fbc, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:19.745849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a139853-8cfd-4331-81c8-e120a6a29400 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:19.815165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1371/2784] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:47:19 2022] PUT /identity/v3/projects/99635c9a2a00410c9df13fa6231c94f9/users/222fd54c252e4f1686690fb1a29d1fbc/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:19.819414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78677924-a8b1-4225-8ae8-faf612331546 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:19.833269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78677924-a8b1-4225-8ae8-faf612331546 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:19.879926 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:19.888120 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73340176-424b-4e18-a513-d2bb5f8633c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:19.888717 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340176-424b-4e18-a513-d2bb5f8633c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:19.888984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340176-424b-4e18-a513-d2bb5f8633c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:19.889253 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340176-424b-4e18-a513-d2bb5f8633c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:19.889907 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73340176-424b-4e18-a513-d2bb5f8633c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:19.892409 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73340176-424b-4e18-a513-d2bb5f8633c8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:19.901779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1414/2785] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:47:19.906043 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a139853-8cfd-4331-81c8-e120a6a29400 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:19.916387 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a139853-8cfd-4331-81c8-e120a6a29400 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:19.920910 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561e287-94b5-415e-bef2-36a305fd3367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:19.921593 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561e287-94b5-415e-bef2-36a305fd3367 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:19.921870 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561e287-94b5-415e-bef2-36a305fd3367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:19.922139 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561e287-94b5-415e-bef2-36a305fd3367 admin admin] PATH_INFO: `/v3/projects/99635c9a2a00410c9df13fa6231c94f9/users/222fd54c252e4f1686690fb1a29d1fbc/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:19.924056 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561e287-94b5-415e-bef2-36a305fd3367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99635c9a2a00410c9df13fa6231c94f9, user_id=222fd54c252e4f1686690fb1a29d1fbc, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:19.926892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561e287-94b5-415e-bef2-36a305fd3367 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:19.936896 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1372/2786] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:47:19 2022] PUT /identity/v3/projects/99635c9a2a00410c9df13fa6231c94f9/users/222fd54c252e4f1686690fb1a29d1fbc/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:20.002199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c940f7-f950-4911-9f3f-99615b83e481 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:20.002488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c940f7-f950-4911-9f3f-99615b83e481 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:20.002771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c940f7-f950-4911-9f3f-99615b83e481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:20.032096 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c0c940f7-f950-4911-9f3f-99615b83e481 None None] MFA Rules not processed for user `222fd54c252e4f1686690fb1a29d1fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:20.033306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c0c940f7-f950-4911-9f3f-99615b83e481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:20.102613 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1415/2787] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:47:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:47:20.116967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561e287-94b5-415e-bef2-36a305fd3367 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:20.118744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d561e287-94b5-415e-bef2-36a305fd3367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:20.175995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561e287-94b5-415e-bef2-36a305fd3367 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:20.179769 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51b50f-0ba7-4616-8800-d8db2628e2a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:20.180366 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51b50f-0ba7-4616-8800-d8db2628e2a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:20.180643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51b50f-0ba7-4616-8800-d8db2628e2a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:20.180915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51b50f-0ba7-4616-8800-d8db2628e2a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:20.235965 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51b50f-0ba7-4616-8800-d8db2628e2a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:20.237399 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51b50f-0ba7-4616-8800-d8db2628e2a6 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:20.246301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1373/2788] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:27.467504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c940f7-f950-4911-9f3f-99615b83e481 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:27.477030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c940f7-f950-4911-9f3f-99615b83e481 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:27.480498 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a673b-5f52-4934-b962-54c940dbcb14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:27.481088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a673b-5f52-4934-b962-54c940dbcb14 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:27.481360 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a673b-5f52-4934-b962-54c940dbcb14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:27.481670 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a673b-5f52-4934-b962-54c940dbcb14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:27.490326 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a673b-5f52-4934-b962-54c940dbcb14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:27.491771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a673b-5f52-4934-b962-54c940dbcb14 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:27.501695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1416/2789] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:30.242194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:37.496893 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=45 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:58.212146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51b50f-0ba7-4616-8800-d8db2628e2a6 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:58.276140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51b50f-0ba7-4616-8800-d8db2628e2a6 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:58.280897 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:58.281539 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:58.281821 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:58.282092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] PATH_INFO: `/v3/users/ca49a389c7b54f4c9ec5160a95bad4c3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:58.283248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca49a389c7b54f4c9ec5160a95bad4c3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:58.285180 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:58.460894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca49a389c7b54f4c9ec5160a95bad4c3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:58.461537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca49a389c7b54f4c9ec5160a95bad4c3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:58.521207 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca49a389c7b54f4c9ec5160a95bad4c3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:58.534799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca49a389c7b54f4c9ec5160a95bad4c3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:58.541567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca49a389c7b54f4c9ec5160a95bad4c3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:58.547660 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1374/2790] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:47:58 2022] DELETE /identity/v3/users/ca49a389c7b54f4c9ec5160a95bad4c3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:58.552184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320881cb-02ca-47b8-b1fd-5af282ee6012 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:58.552704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320881cb-02ca-47b8-b1fd-5af282ee6012 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:58.552812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320881cb-02ca-47b8-b1fd-5af282ee6012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:58.571727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-320881cb-02ca-47b8-b1fd-5af282ee6012 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:58.572922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-320881cb-02ca-47b8-b1fd-5af282ee6012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:58.631295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1417/2791] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:47:58.641478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:58.642874 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:58.699744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255638a-8866-4055-87df-67fccbd0ff58 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:58.704061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017af898-c323-46f5-9d90-106a6e09adbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:58.704664 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af898-c323-46f5-9d90-106a6e09adbd service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:58.704943 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af898-c323-46f5-9d90-106a6e09adbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:58.705217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af898-c323-46f5-9d90-106a6e09adbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:58.714475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017af898-c323-46f5-9d90-106a6e09adbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:58.715846 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017af898-c323-46f5-9d90-106a6e09adbd service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:58.726481 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1375/2792] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:59.094967 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320881cb-02ca-47b8-b1fd-5af282ee6012 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:59.099203 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-320881cb-02ca-47b8-b1fd-5af282ee6012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:59.109661 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320881cb-02ca-47b8-b1fd-5af282ee6012 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:59.114455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:59.115097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:59.115388 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:59.115641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] PATH_INFO: `/v3/projects/caa4ca3167f64aec83e75881392ba739` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:59.116765 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa4ca3167f64aec83e75881392ba739)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:59.118892 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:59.207132 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa4ca3167f64aec83e75881392ba739', 'request_id': 'req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:59.257680 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa4ca3167f64aec83e75881392ba739', 'request_id': 'req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:59.263837 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa4ca3167f64aec83e75881392ba739', 'request_id': 'req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:59.274987 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] The token cache is being invalidate because project caa4ca3167f64aec83e75881392ba739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:47:59.275278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:59.276834 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:59.277420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1418/2793] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:47:59 2022] DELETE /identity/v3/projects/caa4ca3167f64aec83e75881392ba739 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:47:59.282159 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017af898-c323-46f5-9d90-106a6e09adbd service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:59.291952 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017af898-c323-46f5-9d90-106a6e09adbd service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:59.309809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:59.349063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:59.349704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:59.349986 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:59.350253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] PATH_INFO: `/v3/users/222fd54c252e4f1686690fb1a29d1fbc` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:59.351341 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=222fd54c252e4f1686690fb1a29d1fbc)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:59.353374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:59.529053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '222fd54c252e4f1686690fb1a29d1fbc'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:59.529678 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '222fd54c252e4f1686690fb1a29d1fbc'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:59.589520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '222fd54c252e4f1686690fb1a29d1fbc'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:59.601992 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '222fd54c252e4f1686690fb1a29d1fbc'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:59.608806 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-cf25d1a9-8754-4a94-856e-e1618d3066f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '222fd54c252e4f1686690fb1a29d1fbc'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:47:59.614915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1376/2794] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:47:59 2022] DELETE /identity/v3/users/222fd54c252e4f1686690fb1a29d1fbc => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:00.062939 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:00.064562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:00.079133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76923c-8d4e-45fb-a6e2-0b230afe6eee admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:00.144158 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:00.144808 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:00.145506 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:00.145506 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] PATH_INFO: `/v3/projects/99635c9a2a00410c9df13fa6231c94f9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:00.146594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99635c9a2a00410c9df13fa6231c94f9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:00.148507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:00.231490 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:00.248092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99635c9a2a00410c9df13fa6231c94f9', 'request_id': 'req-7af22e80-d511-43ce-9423-30c6b4dd05c4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:00.315317 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99635c9a2a00410c9df13fa6231c94f9', 'request_id': 'req-7af22e80-d511-43ce-9423-30c6b4dd05c4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:00.325196 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99635c9a2a00410c9df13fa6231c94f9', 'request_id': 'req-7af22e80-d511-43ce-9423-30c6b4dd05c4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:00.337364 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] The token cache is being invalidate because project 99635c9a2a00410c9df13fa6231c94f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:48:00.337679 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:00.338646 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7af22e80-d511-43ce-9423-30c6b4dd05c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:00.341444 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1419/2795] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:48:00 2022] DELETE /identity/v3/projects/99635c9a2a00410c9df13fa6231c94f9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:00.347081 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fae12e-8be0-4186-9b7f-4794033423b8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:00.347835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fae12e-8be0-4186-9b7f-4794033423b8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:00.347835 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fae12e-8be0-4186-9b7f-4794033423b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:00.368092 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-99fae12e-8be0-4186-9b7f-4794033423b8 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:00.369312 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-99fae12e-8be0-4186-9b7f-4794033423b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:00.434157 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1377/2796] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:00.442558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356e734-2367-42ba-bafd-6c11c162e916 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:00.442847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356e734-2367-42ba-bafd-6c11c162e916 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:00.443337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356e734-2367-42ba-bafd-6c11c162e916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:00.471476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7356e734-2367-42ba-bafd-6c11c162e916 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:00.471476 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7356e734-2367-42ba-bafd-6c11c162e916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:00.474386 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1420/2797] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:48:00.478533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fae12e-8be0-4186-9b7f-4794033423b8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:00.489185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fae12e-8be0-4186-9b7f-4794033423b8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:00.495840 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1d929-7e3d-4606-aaf5-3191c7508fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:00.500968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1d929-7e3d-4606-aaf5-3191c7508fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:00.500968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1d929-7e3d-4606-aaf5-3191c7508fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:00.500968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1d929-7e3d-4606-aaf5-3191c7508fe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:00.500968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1d929-7e3d-4606-aaf5-3191c7508fe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:48:00.500968 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1d929-7e3d-4606-aaf5-3191c7508fe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:00.502799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1d929-7e3d-4606-aaf5-3191c7508fe6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:00.516110 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1378/2798] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:48:00.520593 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356e734-2367-42ba-bafd-6c11c162e916 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:00.532329 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356e734-2367-42ba-bafd-6c11c162e916 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:00.539055 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b156af-d0b5-4115-a8d8-610c7b7de92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:00.539663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b156af-d0b5-4115-a8d8-610c7b7de92c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:00.539938 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b156af-d0b5-4115-a8d8-610c7b7de92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:00.540204 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b156af-d0b5-4115-a8d8-610c7b7de92c admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:00.540824 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b156af-d0b5-4115-a8d8-610c7b7de92c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:00.542765 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b156af-d0b5-4115-a8d8-610c7b7de92c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:00.641134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1421/2799] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:48:00 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:00.642309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1d929-7e3d-4606-aaf5-3191c7508fe6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:00.652471 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1d929-7e3d-4606-aaf5-3191c7508fe6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:00.678947 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:00.710699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda39af0-adb1-4a5c-b98f-f3923cfafec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:00.711309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda39af0-adb1-4a5c-b98f-f3923cfafec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:00.711589 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda39af0-adb1-4a5c-b98f-f3923cfafec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:00.711860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda39af0-adb1-4a5c-b98f-f3923cfafec5 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:00.712490 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda39af0-adb1-4a5c-b98f-f3923cfafec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:00.714673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda39af0-adb1-4a5c-b98f-f3923cfafec5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:00.795810 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1379/2800] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:48:00 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:48:00.799667 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b156af-d0b5-4115-a8d8-610c7b7de92c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:00.810100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b156af-d0b5-4115-a8d8-610c7b7de92c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:00.814960 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6afe2-61b3-4d5c-ad2b-bbbe2e9b51a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:00.815550 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6afe2-61b3-4d5c-ad2b-bbbe2e9b51a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:00.815823 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6afe2-61b3-4d5c-ad2b-bbbe2e9b51a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:00.816089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6afe2-61b3-4d5c-ad2b-bbbe2e9b51a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:00.816723 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6afe2-61b3-4d5c-ad2b-bbbe2e9b51a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:00.818330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6afe2-61b3-4d5c-ad2b-bbbe2e9b51a5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:00.829904 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1422/2801] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:00.833951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda39af0-adb1-4a5c-b98f-f3923cfafec5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:00.844579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda39af0-adb1-4a5c-b98f-f3923cfafec5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:00.849504 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcab734-b1c1-4f9d-a19a-1ab5cc58e659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:00.850100 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcab734-b1c1-4f9d-a19a-1ab5cc58e659 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:00.850381 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcab734-b1c1-4f9d-a19a-1ab5cc58e659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:00.850689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcab734-b1c1-4f9d-a19a-1ab5cc58e659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:00.851548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcab734-b1c1-4f9d-a19a-1ab5cc58e659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:00.853178 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcab734-b1c1-4f9d-a19a-1ab5cc58e659 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:00.863783 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1380/2802] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:48:00.867989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6afe2-61b3-4d5c-ad2b-bbbe2e9b51a5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:00.878572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6afe2-61b3-4d5c-ad2b-bbbe2e9b51a5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:00.883331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2391fe7-441f-4437-8432-e9bba519bbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:00.883976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2391fe7-441f-4437-8432-e9bba519bbdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:00.884257 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2391fe7-441f-4437-8432-e9bba519bbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:00.884531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2391fe7-441f-4437-8432-e9bba519bbdd admin admin] PATH_INFO: `/v3/projects/0dde47a904944e3bb95f3da3ffcf9300/users/1ebcb6c605c64bbcaca21ed006c8990f/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:00.900606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2391fe7-441f-4437-8432-e9bba519bbdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dde47a904944e3bb95f3da3ffcf9300, user_id=1ebcb6c605c64bbcaca21ed006c8990f, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:00.902511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2391fe7-441f-4437-8432-e9bba519bbdd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:00.973789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1423/2803] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:00 2022] PUT /identity/v3/projects/0dde47a904944e3bb95f3da3ffcf9300/users/1ebcb6c605c64bbcaca21ed006c8990f/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:48:00.985780 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24089de8-da5b-4e15-ab3e-f311abaec4d4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:00.986070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24089de8-da5b-4e15-ab3e-f311abaec4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:00.986344 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24089de8-da5b-4e15-ab3e-f311abaec4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:01.017413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-24089de8-da5b-4e15-ab3e-f311abaec4d4 None None] MFA Rules not processed for user `1ebcb6c605c64bbcaca21ed006c8990f`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:01.018672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-24089de8-da5b-4e15-ab3e-f311abaec4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:01.087333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1381/2804] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:01.092094 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2391fe7-441f-4437-8432-e9bba519bbdd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:01.159473 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2391fe7-441f-4437-8432-e9bba519bbdd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:01.164168 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7775d3-f340-4e1b-ada6-772e0d591094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:01.164771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7775d3-f340-4e1b-ada6-772e0d591094 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:01.165048 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7775d3-f340-4e1b-ada6-772e0d591094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:01.165322 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7775d3-f340-4e1b-ada6-772e0d591094 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:01.165989 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7775d3-f340-4e1b-ada6-772e0d591094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:01.167868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7775d3-f340-4e1b-ada6-772e0d591094 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:01.288455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1424/2805] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:48:01 2022] POST /identity/v3/projects => generated 381 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:01.292535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24089de8-da5b-4e15-ab3e-f311abaec4d4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:01.303641 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24089de8-da5b-4e15-ab3e-f311abaec4d4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:01.360781 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836660b-cedc-40ea-9107-eb290ea6bc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:01.361496 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836660b-cedc-40ea-9107-eb290ea6bc75 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:01.361669 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836660b-cedc-40ea-9107-eb290ea6bc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:01.361939 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836660b-cedc-40ea-9107-eb290ea6bc75 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:01.362554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836660b-cedc-40ea-9107-eb290ea6bc75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:01.364459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836660b-cedc-40ea-9107-eb290ea6bc75 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:01.487551 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1382/2806] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:48:01 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:01.491488 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7775d3-f340-4e1b-ada6-772e0d591094 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:01.502368 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7775d3-f340-4e1b-ada6-772e0d591094 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:01.506993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d306b2b-0df4-44df-9f54-c0a4ac3a50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:01.507585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d306b2b-0df4-44df-9f54-c0a4ac3a50b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:01.507867 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d306b2b-0df4-44df-9f54-c0a4ac3a50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:01.508138 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d306b2b-0df4-44df-9f54-c0a4ac3a50b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:01.508776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d306b2b-0df4-44df-9f54-c0a4ac3a50b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:01.510394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d306b2b-0df4-44df-9f54-c0a4ac3a50b3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:01.520783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1425/2807] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:01.524799 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836660b-cedc-40ea-9107-eb290ea6bc75 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:01.534649 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836660b-cedc-40ea-9107-eb290ea6bc75 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:01.539412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeb00b-ee9d-4701-9f00-f44c54cb71bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:01.540052 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb00b-ee9d-4701-9f00-f44c54cb71bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:01.540325 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb00b-ee9d-4701-9f00-f44c54cb71bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:01.540594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb00b-ee9d-4701-9f00-f44c54cb71bd admin admin] PATH_INFO: `/v3/projects/b4bec7a36c7c47879bc128ee6ef1f34c/users/d1cef9348f664cd1a0ca938fce4939a8/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:01.555998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeb00b-ee9d-4701-9f00-f44c54cb71bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4bec7a36c7c47879bc128ee6ef1f34c, user_id=d1cef9348f664cd1a0ca938fce4939a8, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:01.559133 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeb00b-ee9d-4701-9f00-f44c54cb71bd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:01.617717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1383/2808] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:01 2022] PUT /identity/v3/projects/b4bec7a36c7c47879bc128ee6ef1f34c/users/d1cef9348f664cd1a0ca938fce4939a8/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:48:01.621691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d306b2b-0df4-44df-9f54-c0a4ac3a50b3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:01.632438 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d306b2b-0df4-44df-9f54-c0a4ac3a50b3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:01.652244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:01.689662 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990a194-1834-4313-9b57-ed4442577605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:01.690293 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990a194-1834-4313-9b57-ed4442577605 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:01.690573 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990a194-1834-4313-9b57-ed4442577605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:01.690845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990a194-1834-4313-9b57-ed4442577605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:01.691475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990a194-1834-4313-9b57-ed4442577605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:01.693134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990a194-1834-4313-9b57-ed4442577605 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:01.703732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1426/2809] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:01.707412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeb00b-ee9d-4701-9f00-f44c54cb71bd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:01.712936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:01.718982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeb00b-ee9d-4701-9f00-f44c54cb71bd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:01.723893 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0162cd2-9549-4cbe-aa46-11780a25dd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:01.724546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0162cd2-9549-4cbe-aa46-11780a25dd4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:01.725101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0162cd2-9549-4cbe-aa46-11780a25dd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:01.725330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0162cd2-9549-4cbe-aa46-11780a25dd4a admin admin] PATH_INFO: `/v3/projects/b4bec7a36c7c47879bc128ee6ef1f34c/users/d1cef9348f664cd1a0ca938fce4939a8/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:01.727307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0162cd2-9549-4cbe-aa46-11780a25dd4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4bec7a36c7c47879bc128ee6ef1f34c, user_id=d1cef9348f664cd1a0ca938fce4939a8, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:01.734886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0162cd2-9549-4cbe-aa46-11780a25dd4a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:01.741598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1384/2810] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:01 2022] PUT /identity/v3/projects/b4bec7a36c7c47879bc128ee6ef1f34c/users/d1cef9348f664cd1a0ca938fce4939a8/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:01.751813 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e28a44-d142-46fa-9a0a-058cae12a408 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:01.752101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e28a44-d142-46fa-9a0a-058cae12a408 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:01.752377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e28a44-d142-46fa-9a0a-058cae12a408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:01.780064 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b7e28a44-d142-46fa-9a0a-058cae12a408 None None] MFA Rules not processed for user `d1cef9348f664cd1a0ca938fce4939a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:01.781840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b7e28a44-d142-46fa-9a0a-058cae12a408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:01.848850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1427/2811] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:01.861792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0162cd2-9549-4cbe-aa46-11780a25dd4a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:01.863518 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f0162cd2-9549-4cbe-aa46-11780a25dd4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:01.921022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0162cd2-9549-4cbe-aa46-11780a25dd4a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:01.925542 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129e530-8846-4d37-83bd-fe91dfc0555f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:01.926171 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129e530-8846-4d37-83bd-fe91dfc0555f service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:01.926453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129e530-8846-4d37-83bd-fe91dfc0555f service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:01.926838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129e530-8846-4d37-83bd-fe91dfc0555f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:01.935946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129e530-8846-4d37-83bd-fe91dfc0555f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:01.937550 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129e530-8846-4d37-83bd-fe91dfc0555f service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:01.947130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1385/2812] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:02.751021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e28a44-d142-46fa-9a0a-058cae12a408 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:02.760380 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e28a44-d142-46fa-9a0a-058cae12a408 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:02.764998 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c72c86-9662-47ae-bc8c-e750c31cd241 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1cef9348f664cd1a0ca938fce4939a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4bec7a36c7c47879bc128ee6ef1f34c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:02.765735 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c72c86-9662-47ae-bc8c-e750c31cd241 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:02.766077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c72c86-9662-47ae-bc8c-e750c31cd241 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:02.766353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c72c86-9662-47ae-bc8c-e750c31cd241 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:02.767004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c72c86-9662-47ae-bc8c-e750c31cd241 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:48:02.767309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c72c86-9662-47ae-bc8c-e750c31cd241 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:02.768931 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c72c86-9662-47ae-bc8c-e750c31cd241 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:02.783585 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1428/2813] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:48:02.787918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129e530-8846-4d37-83bd-fe91dfc0555f service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:02.798279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129e530-8846-4d37-83bd-fe91dfc0555f service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:02.803291 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d518836-c8b8-4f1f-8d51-33b1ea05f405 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1cef9348f664cd1a0ca938fce4939a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4bec7a36c7c47879bc128ee6ef1f34c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:02.806613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d518836-c8b8-4f1f-8d51-33b1ea05f405 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:02.806613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d518836-c8b8-4f1f-8d51-33b1ea05f405 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:02.806613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d518836-c8b8-4f1f-8d51-33b1ea05f405 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:02.806613 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d518836-c8b8-4f1f-8d51-33b1ea05f405 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:02.807365 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d518836-c8b8-4f1f-8d51-33b1ea05f405 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:02.916891 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1386/2814] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:48:02 2022] POST /identity/v3/projects => generated 353 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:02.921210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c72c86-9662-47ae-bc8c-e750c31cd241 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:02.933174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c72c86-9662-47ae-bc8c-e750c31cd241 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:02.991114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf875c-db12-4c89-aa07-b9e9870ac028 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1cef9348f664cd1a0ca938fce4939a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4bec7a36c7c47879bc128ee6ef1f34c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:02.991640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf875c-db12-4c89-aa07-b9e9870ac028 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:02.991922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf875c-db12-4c89-aa07-b9e9870ac028 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:02.992198 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf875c-db12-4c89-aa07-b9e9870ac028 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:02.992840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf875c-db12-4c89-aa07-b9e9870ac028 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:48:02.993144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf875c-db12-4c89-aa07-b9e9870ac028 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:02.994770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf875c-db12-4c89-aa07-b9e9870ac028 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:03.006421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1429/2815] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:48:03.043229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d518836-c8b8-4f1f-8d51-33b1ea05f405 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:03.054437 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d518836-c8b8-4f1f-8d51-33b1ea05f405 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:03.059131 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1cef9348f664cd1a0ca938fce4939a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4bec7a36c7c47879bc128ee6ef1f34c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:03.059745 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:03.060030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:03.060299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] PATH_INFO: `/v3/projects/81e3434d10cb492f891e13ec756eda09` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:03.061427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: Authorizing `identity:delete_project(project_id=81e3434d10cb492f891e13ec756eda09)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:03.064016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:03.190668 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e3434d10cb492f891e13ec756eda09', 'request_id': 'req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:03.246856 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e3434d10cb492f891e13ec756eda09', 'request_id': 'req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:03.254697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e3434d10cb492f891e13ec756eda09', 'request_id': 'req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:03.266057 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] The token cache is being invalidate because project 81e3434d10cb492f891e13ec756eda09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:48:03.266383 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:03.267037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bbafbbc1-a5fb-46db-a905-e6d8d58ae4c5 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:03.268517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1387/2816] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:48:03 2022] DELETE /identity/v3/projects/81e3434d10cb492f891e13ec756eda09 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:03.849727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf875c-db12-4c89-aa07-b9e9870ac028 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:03.851420 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0cbf875c-db12-4c89-aa07-b9e9870ac028 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:03.885101 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:03.920719 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf875c-db12-4c89-aa07-b9e9870ac028 tempest-VolumeQuotaClassesTest-1912147313 tempest-VolumeQuotaClassesTest-1912147313-project-admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:03.980431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:03.981075 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:03.981350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:03.981644 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] PATH_INFO: `/v3/users/1ebcb6c605c64bbcaca21ed006c8990f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:03.983184 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ebcb6c605c64bbcaca21ed006c8990f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:03.986220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:04.197324 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ebcb6c605c64bbcaca21ed006c8990f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:04.209794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ebcb6c605c64bbcaca21ed006c8990f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:04.216607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ebcb6c605c64bbcaca21ed006c8990f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:04.280736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ebcb6c605c64bbcaca21ed006c8990f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:04.281450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ebcb6c605c64bbcaca21ed006c8990f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:04.287618 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1430/2817] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:48:03 2022] DELETE /identity/v3/users/1ebcb6c605c64bbcaca21ed006c8990f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:04.292094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e73a2-841b-4d95-8867-ce4c79d35a65 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:04.292591 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e73a2-841b-4d95-8867-ce4c79d35a65 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:04.292707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e73a2-841b-4d95-8867-ce4c79d35a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:04.311413 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8c7e73a2-841b-4d95-8867-ce4c79d35a65 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:04.313140 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8c7e73a2-841b-4d95-8867-ce4c79d35a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:04.360837 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:04.372717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1388/2818] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:48:04.383543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:04.384930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:04.442809 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca006c-10d8-4a2f-9181-7f677712deb1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:04.446654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677cbbe-5241-4a6f-9c33-286d0a076f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:04.447399 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677cbbe-5241-4a6f-9c33-286d0a076f8c service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:04.447624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677cbbe-5241-4a6f-9c33-286d0a076f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:04.447891 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677cbbe-5241-4a6f-9c33-286d0a076f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:04.455985 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:04.458135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677cbbe-5241-4a6f-9c33-286d0a076f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:04.459541 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677cbbe-5241-4a6f-9c33-286d0a076f8c service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:04.469687 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1431/2819] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:04.848473 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e73a2-841b-4d95-8867-ce4c79d35a65 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:04.849919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8c7e73a2-841b-4d95-8867-ce4c79d35a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:04.862554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e73a2-841b-4d95-8867-ce4c79d35a65 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:04.867203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:04.867838 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:04.868108 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:04.868378 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] PATH_INFO: `/v3/projects/0dde47a904944e3bb95f3da3ffcf9300` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:04.869520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dde47a904944e3bb95f3da3ffcf9300)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:04.871431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:04.972695 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dde47a904944e3bb95f3da3ffcf9300', 'request_id': 'req-135667cc-e649-4a7d-817b-c00d89613635'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.040714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dde47a904944e3bb95f3da3ffcf9300', 'request_id': 'req-135667cc-e649-4a7d-817b-c00d89613635'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.047836 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dde47a904944e3bb95f3da3ffcf9300', 'request_id': 'req-135667cc-e649-4a7d-817b-c00d89613635'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.058460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] The token cache is being invalidate because project 0dde47a904944e3bb95f3da3ffcf9300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:48:05.058743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.059374 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.060833 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1389/2820] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:48:04 2022] DELETE /identity/v3/projects/0dde47a904944e3bb95f3da3ffcf9300 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:05.065721 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677cbbe-5241-4a6f-9c33-286d0a076f8c service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:05.075631 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677cbbe-5241-4a6f-9c33-286d0a076f8c service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:05.132455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:05.133085 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:05.133365 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:05.133676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] PATH_INFO: `/v3/users/d1cef9348f664cd1a0ca938fce4939a8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:05.134788 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1cef9348f664cd1a0ca938fce4939a8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:05.136692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:05.300143 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1cef9348f664cd1a0ca938fce4939a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.312727 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1cef9348f664cd1a0ca938fce4939a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.319607 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1cef9348f664cd1a0ca938fce4939a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.372030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1cef9348f664cd1a0ca938fce4939a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.372663 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-263b32bf-969a-406b-9e4f-0e2e88e823d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1cef9348f664cd1a0ca938fce4939a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.379414 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1432/2821] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:48:05 2022] DELETE /identity/v3/users/d1cef9348f664cd1a0ca938fce4939a8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:05.756946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:05.758461 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:05.773693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135667cc-e649-4a7d-817b-c00d89613635 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:05.842210 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:05.842849 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:05.843679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:05.844160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] PATH_INFO: `/v3/projects/b4bec7a36c7c47879bc128ee6ef1f34c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:05.845433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4bec7a36c7c47879bc128ee6ef1f34c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:05.847280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:05.937587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4bec7a36c7c47879bc128ee6ef1f34c', 'request_id': 'req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.988975 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4bec7a36c7c47879bc128ee6ef1f34c', 'request_id': 'req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:05.995253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4bec7a36c7c47879bc128ee6ef1f34c', 'request_id': 'req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:06.006167 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] The token cache is being invalidate because project b4bec7a36c7c47879bc128ee6ef1f34c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:48:06.006455 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:06.007093 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f33aa209-ae0b-4b80-bf70-2d8d5f64c9dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:06.008574 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1390/2822] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:48:05 2022] DELETE /identity/v3/projects/b4bec7a36c7c47879bc128ee6ef1f34c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:06.014347 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512814a-0452-41ec-af18-91246e7767b7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.014658 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512814a-0452-41ec-af18-91246e7767b7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.014976 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512814a-0452-41ec-af18-91246e7767b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.035879 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8512814a-0452-41ec-af18-91246e7767b7 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:06.037077 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8512814a-0452-41ec-af18-91246e7767b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:06.096024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1433/2823] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:06.104826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b59c3c-1719-4007-bdce-e8b01b9e1805 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.105111 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b59c3c-1719-4007-bdce-e8b01b9e1805 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.105385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b59c3c-1719-4007-bdce-e8b01b9e1805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.121999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:06.128639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-56b59c3c-1719-4007-bdce-e8b01b9e1805 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:06.130823 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-56b59c3c-1719-4007-bdce-e8b01b9e1805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:06.139651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1391/2824] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:06.145255 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512814a-0452-41ec-af18-91246e7767b7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:06.154915 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512814a-0452-41ec-af18-91246e7767b7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:06.160140 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc68bf7-8c8f-429a-91d9-32647665f1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:06.160812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68bf7-8c8f-429a-91d9-32647665f1cd admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.161020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68bf7-8c8f-429a-91d9-32647665f1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.161290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68bf7-8c8f-429a-91d9-32647665f1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.161973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc68bf7-8c8f-429a-91d9-32647665f1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:48:06.162274 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc68bf7-8c8f-429a-91d9-32647665f1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:06.163858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc68bf7-8c8f-429a-91d9-32647665f1cd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:06.175689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1434/2825] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:48:06.180089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b59c3c-1719-4007-bdce-e8b01b9e1805 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:06.190494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b59c3c-1719-4007-bdce-e8b01b9e1805 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:06.195224 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d638b2-6ae9-4254-ab59-c74f0ce16438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:06.196067 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d638b2-6ae9-4254-ab59-c74f0ce16438 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.196340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d638b2-6ae9-4254-ab59-c74f0ce16438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.196610 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d638b2-6ae9-4254-ab59-c74f0ce16438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.197225 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d638b2-6ae9-4254-ab59-c74f0ce16438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:06.199156 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d638b2-6ae9-4254-ab59-c74f0ce16438 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:06.259583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1392/2826] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:48:06 2022] POST /identity/v3/projects => generated 365 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:06.263935 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc68bf7-8c8f-429a-91d9-32647665f1cd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:06.269789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:06.274888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc68bf7-8c8f-429a-91d9-32647665f1cd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:06.332353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579d85d-0ff2-48d9-b878-40b3c66ec9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:06.332970 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579d85d-0ff2-48d9-b878-40b3c66ec9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.333234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579d85d-0ff2-48d9-b878-40b3c66ec9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.333535 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579d85d-0ff2-48d9-b878-40b3c66ec9c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.334197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579d85d-0ff2-48d9-b878-40b3c66ec9c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:06.336093 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579d85d-0ff2-48d9-b878-40b3c66ec9c1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:06.430052 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1435/2827] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:48:06 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:06.434279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d638b2-6ae9-4254-ab59-c74f0ce16438 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:06.445204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d638b2-6ae9-4254-ab59-c74f0ce16438 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:06.450026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc8625-e142-4b40-95e9-64687482b5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:06.455063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc8625-e142-4b40-95e9-64687482b5fe admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.455063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc8625-e142-4b40-95e9-64687482b5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.455063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc8625-e142-4b40-95e9-64687482b5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.455063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc8625-e142-4b40-95e9-64687482b5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:06.455063 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc8625-e142-4b40-95e9-64687482b5fe admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:06.467457 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1393/2828] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:06.471674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579d85d-0ff2-48d9-b878-40b3c66ec9c1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:06.481624 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579d85d-0ff2-48d9-b878-40b3c66ec9c1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:06.486514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce91db-6f31-4c01-9ab7-67e453f6aee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:06.487180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce91db-6f31-4c01-9ab7-67e453f6aee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.487457 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce91db-6f31-4c01-9ab7-67e453f6aee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.487828 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce91db-6f31-4c01-9ab7-67e453f6aee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.488654 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce91db-6f31-4c01-9ab7-67e453f6aee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:06.490467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce91db-6f31-4c01-9ab7-67e453f6aee3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:06.500767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1436/2829] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:06.505723 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc8625-e142-4b40-95e9-64687482b5fe admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:06.515701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc8625-e142-4b40-95e9-64687482b5fe admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:06.521239 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cf21e-efc0-4251-8116-3eb72cd40d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:06.521911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cf21e-efc0-4251-8116-3eb72cd40d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.522190 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cf21e-efc0-4251-8116-3eb72cd40d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.522459 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cf21e-efc0-4251-8116-3eb72cd40d2a admin admin] PATH_INFO: `/v3/projects/e2e91c6e7f6a4e05b7f19fa9e4c7be90/users/95d7f645ac0b4471a217793a7a9926bf/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.537251 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cf21e-efc0-4251-8116-3eb72cd40d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e91c6e7f6a4e05b7f19fa9e4c7be90, user_id=95d7f645ac0b4471a217793a7a9926bf, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:06.539930 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cf21e-efc0-4251-8116-3eb72cd40d2a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:06.606809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1394/2830] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:06 2022] PUT /identity/v3/projects/e2e91c6e7f6a4e05b7f19fa9e4c7be90/users/95d7f645ac0b4471a217793a7a9926bf/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:06.617556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07403052-aadd-43f4-851d-4d000effd32a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.617840 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07403052-aadd-43f4-851d-4d000effd32a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.618113 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07403052-aadd-43f4-851d-4d000effd32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.645955 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-07403052-aadd-43f4-851d-4d000effd32a None None] MFA Rules not processed for user `95d7f645ac0b4471a217793a7a9926bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:06.647144 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-07403052-aadd-43f4-851d-4d000effd32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:06.711172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1437/2831] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:06.715605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cf21e-efc0-4251-8116-3eb72cd40d2a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:06.778379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cf21e-efc0-4251-8116-3eb72cd40d2a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:06.783172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f432fe9-db7f-4c02-bb8f-cb6b9fa6a312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:06.783768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f432fe9-db7f-4c02-bb8f-cb6b9fa6a312 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.784048 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f432fe9-db7f-4c02-bb8f-cb6b9fa6a312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.784327 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f432fe9-db7f-4c02-bb8f-cb6b9fa6a312 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.784960 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f432fe9-db7f-4c02-bb8f-cb6b9fa6a312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:06.786967 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f432fe9-db7f-4c02-bb8f-cb6b9fa6a312 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:06.870672 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1395/2832] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:48:06 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:06.874771 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07403052-aadd-43f4-851d-4d000effd32a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:06.886330 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07403052-aadd-43f4-851d-4d000effd32a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:06.944105 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83ea49-7fe1-4e51-8afc-93ae0969e94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:06.944713 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83ea49-7fe1-4e51-8afc-93ae0969e94c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:06.944997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83ea49-7fe1-4e51-8afc-93ae0969e94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:06.945272 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83ea49-7fe1-4e51-8afc-93ae0969e94c admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:06.946191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83ea49-7fe1-4e51-8afc-93ae0969e94c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:06.948092 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83ea49-7fe1-4e51-8afc-93ae0969e94c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:06.995208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:07.000097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1438/2833] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:48:06 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:48:07.004247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f432fe9-db7f-4c02-bb8f-cb6b9fa6a312 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:07.014946 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f432fe9-db7f-4c02-bb8f-cb6b9fa6a312 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:07.019744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262128d-8c0f-4e7c-bb8d-6d02bac45c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:07.020367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262128d-8c0f-4e7c-bb8d-6d02bac45c45 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:07.020645 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262128d-8c0f-4e7c-bb8d-6d02bac45c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:07.020918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262128d-8c0f-4e7c-bb8d-6d02bac45c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:07.021588 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262128d-8c0f-4e7c-bb8d-6d02bac45c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:07.023183 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262128d-8c0f-4e7c-bb8d-6d02bac45c45 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:07.034453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1396/2834] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:07.038400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83ea49-7fe1-4e51-8afc-93ae0969e94c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:07.049877 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83ea49-7fe1-4e51-8afc-93ae0969e94c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:07.054475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153ffac-75b2-408c-895d-d16b15eefc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:07.055142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153ffac-75b2-408c-895d-d16b15eefc73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:07.055421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153ffac-75b2-408c-895d-d16b15eefc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:07.055691 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153ffac-75b2-408c-895d-d16b15eefc73 admin admin] PATH_INFO: `/v3/projects/f3dd41f3c3744ae497e74bf1095c94b8/users/9b8bd92ed0274cb2b9675b92196369f7/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:07.070697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153ffac-75b2-408c-895d-d16b15eefc73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3dd41f3c3744ae497e74bf1095c94b8, user_id=9b8bd92ed0274cb2b9675b92196369f7, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:07.073281 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153ffac-75b2-408c-895d-d16b15eefc73 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:07.166173 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1439/2835] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:07 2022] PUT /identity/v3/projects/f3dd41f3c3744ae497e74bf1095c94b8/users/9b8bd92ed0274cb2b9675b92196369f7/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:48:07.170028 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262128d-8c0f-4e7c-bb8d-6d02bac45c45 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:07.180761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262128d-8c0f-4e7c-bb8d-6d02bac45c45 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:07.237906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6888888-3ebf-4288-b35f-40bad2e3a712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:07.238671 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6888888-3ebf-4288-b35f-40bad2e3a712 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:07.238982 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6888888-3ebf-4288-b35f-40bad2e3a712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:07.239253 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6888888-3ebf-4288-b35f-40bad2e3a712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:07.239889 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6888888-3ebf-4288-b35f-40bad2e3a712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:07.241515 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6888888-3ebf-4288-b35f-40bad2e3a712 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:07.251843 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1397/2836] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:07.255930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153ffac-75b2-408c-895d-d16b15eefc73 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:07.271131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153ffac-75b2-408c-895d-d16b15eefc73 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:07.275701 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62707223-5807-4946-ac4c-119c154960bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:07.276345 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62707223-5807-4946-ac4c-119c154960bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:07.276625 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62707223-5807-4946-ac4c-119c154960bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:07.276888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62707223-5807-4946-ac4c-119c154960bb admin admin] PATH_INFO: `/v3/projects/f3dd41f3c3744ae497e74bf1095c94b8/users/9b8bd92ed0274cb2b9675b92196369f7/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:07.278816 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62707223-5807-4946-ac4c-119c154960bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3dd41f3c3744ae497e74bf1095c94b8, user_id=9b8bd92ed0274cb2b9675b92196369f7, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:07.281433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62707223-5807-4946-ac4c-119c154960bb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:07.292529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1440/2837] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:07 2022] PUT /identity/v3/projects/f3dd41f3c3744ae497e74bf1095c94b8/users/9b8bd92ed0274cb2b9675b92196369f7/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:07.303203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b9cdc-5d6b-43c8-8796-5d72b8c87921 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:07.303491 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b9cdc-5d6b-43c8-8796-5d72b8c87921 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:07.303790 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b9cdc-5d6b-43c8-8796-5d72b8c87921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:07.330282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:07.335663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fb5b9cdc-5d6b-43c8-8796-5d72b8c87921 None None] MFA Rules not processed for user `9b8bd92ed0274cb2b9675b92196369f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:07.335663 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fb5b9cdc-5d6b-43c8-8796-5d72b8c87921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:07.404221 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1398/2838] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:07.416024 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62707223-5807-4946-ac4c-119c154960bb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:07.417730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-62707223-5807-4946-ac4c-119c154960bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:07.475948 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62707223-5807-4946-ac4c-119c154960bb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:07.479709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82e344-492c-49f9-9492-9426a918c149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:07.480309 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82e344-492c-49f9-9492-9426a918c149 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:07.480590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82e344-492c-49f9-9492-9426a918c149 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:07.480864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82e344-492c-49f9-9492-9426a918c149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:07.489932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82e344-492c-49f9-9492-9426a918c149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:07.491350 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82e344-492c-49f9-9492-9426a918c149 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:07.500672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1441/2839] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:07.673839 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b9cdc-5d6b-43c8-8796-5d72b8c87921 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:07.683527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b9cdc-5d6b-43c8-8796-5d72b8c87921 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:07.687149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b616-c6ef-43bb-9efa-550e7469669b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:07.687744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b616-c6ef-43bb-9efa-550e7469669b service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:07.688021 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b616-c6ef-43bb-9efa-550e7469669b service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:07.688798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b616-c6ef-43bb-9efa-550e7469669b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:07.747278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b616-c6ef-43bb-9efa-550e7469669b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:07.748718 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b616-c6ef-43bb-9efa-550e7469669b service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:07.758852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1399/2840] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:48:13.998582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82e344-492c-49f9-9492-9426a918c149 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:14.060036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82e344-492c-49f9-9492-9426a918c149 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:14.065000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:14.065656 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:14.065937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:14.066197 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] PATH_INFO: `/v3/users/95d7f645ac0b4471a217793a7a9926bf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:14.067295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d7f645ac0b4471a217793a7a9926bf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:14.069188 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:14.242733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d7f645ac0b4471a217793a7a9926bf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:14.254622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d7f645ac0b4471a217793a7a9926bf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:14.256711 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:14.261986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d7f645ac0b4471a217793a7a9926bf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:14.326858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d7f645ac0b4471a217793a7a9926bf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:14.327529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d7f645ac0b4471a217793a7a9926bf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:14.335248 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1442/2841] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:48:13 2022] DELETE /identity/v3/users/95d7f645ac0b4471a217793a7a9926bf => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:14.339887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5374-4d7a-471e-b909-f03b8d2998d9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:14.340354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5374-4d7a-471e-b909-f03b8d2998d9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:14.340453 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5374-4d7a-471e-b909-f03b8d2998d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:14.360911 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9dbc5374-4d7a-471e-b909-f03b8d2998d9 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:14.362155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9dbc5374-4d7a-471e-b909-f03b8d2998d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:14.423525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1400/2842] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:14.433340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:14.434794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:14.499082 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f449c35-4149-4d46-8c5e-b3332ebd803d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:14.503417 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410b98-3fa2-4566-9011-64f61b8916bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:14.504030 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410b98-3fa2-4566-9011-64f61b8916bd service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:14.504311 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410b98-3fa2-4566-9011-64f61b8916bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:14.504590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410b98-3fa2-4566-9011-64f61b8916bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:14.515636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6410b98-3fa2-4566-9011-64f61b8916bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:14.517937 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6410b98-3fa2-4566-9011-64f61b8916bd service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:14.528710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1443/2843] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:14.962724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc5374-4d7a-471e-b909-f03b8d2998d9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:14.964151 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9dbc5374-4d7a-471e-b909-f03b8d2998d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:14.976548 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc5374-4d7a-471e-b909-f03b8d2998d9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:14.981335 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:14.982010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:14.982291 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:14.982567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] PATH_INFO: `/v3/projects/e2e91c6e7f6a4e05b7f19fa9e4c7be90` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:14.983703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2e91c6e7f6a4e05b7f19fa9e4c7be90)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:14.986364 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:15.087079 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:15.099357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2e91c6e7f6a4e05b7f19fa9e4c7be90', 'request_id': 'req-030a82d4-d08b-4e99-b9eb-44c332ae00c7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:15.155065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2e91c6e7f6a4e05b7f19fa9e4c7be90', 'request_id': 'req-030a82d4-d08b-4e99-b9eb-44c332ae00c7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:15.166288 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2e91c6e7f6a4e05b7f19fa9e4c7be90', 'request_id': 'req-030a82d4-d08b-4e99-b9eb-44c332ae00c7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:15.179196 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] The token cache is being invalidate because project e2e91c6e7f6a4e05b7f19fa9e4c7be90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:48:15.179479 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:15.180191 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:15.181729 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1401/2844] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:48:14 2022] DELETE /identity/v3/projects/e2e91c6e7f6a4e05b7f19fa9e4c7be90 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:15.186131 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410b98-3fa2-4566-9011-64f61b8916bd service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:15.201114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410b98-3fa2-4566-9011-64f61b8916bd service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:15.259262 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:15.259894 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:15.260174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:15.260445 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] PATH_INFO: `/v3/users/9b8bd92ed0274cb2b9675b92196369f7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:15.261602 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b8bd92ed0274cb2b9675b92196369f7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:15.263578 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:15.421638 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b8bd92ed0274cb2b9675b92196369f7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:15.434710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b8bd92ed0274cb2b9675b92196369f7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:15.441922 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b8bd92ed0274cb2b9675b92196369f7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:15.497130 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b8bd92ed0274cb2b9675b92196369f7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:15.497782 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-56fd4e32-ac17-4237-a44e-cfe124930b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b8bd92ed0274cb2b9675b92196369f7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:15.504047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1444/2845] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:48:15 2022] DELETE /identity/v3/users/9b8bd92ed0274cb2b9675b92196369f7 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:15.939326 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:15.940764 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:15.955317 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a82d4-d08b-4e99-b9eb-44c332ae00c7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:16.012576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:16.013230 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:16.013535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:16.013805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] PATH_INFO: `/v3/projects/f3dd41f3c3744ae497e74bf1095c94b8` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:16.014914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3dd41f3c3744ae497e74bf1095c94b8)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:16.017406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:16.132075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3dd41f3c3744ae497e74bf1095c94b8', 'request_id': 'req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:16.194579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3dd41f3c3744ae497e74bf1095c94b8', 'request_id': 'req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:16.201336 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3dd41f3c3744ae497e74bf1095c94b8', 'request_id': 'req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:16.212768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] The token cache is being invalidate because project f3dd41f3c3744ae497e74bf1095c94b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:48:16.213059 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:16.213782 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-473420e2-6db5-4cf5-a9a9-d8f74ce89dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:16.215194 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1402/2846] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:48:15 2022] DELETE /identity/v3/projects/f3dd41f3c3744ae497e74bf1095c94b8 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:16.220376 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e397df4-92fa-46d2-bfbf-303de62da97d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:16.220868 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e397df4-92fa-46d2-bfbf-303de62da97d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:16.220993 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e397df4-92fa-46d2-bfbf-303de62da97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:16.234689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:16.240986 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2e397df4-92fa-46d2-bfbf-303de62da97d None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:16.242239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2e397df4-92fa-46d2-bfbf-303de62da97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:16.300930 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1445/2847] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:16.308956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aba1a2-67e8-4b83-8b4e-fc191dec0cc5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:16.309248 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aba1a2-67e8-4b83-8b4e-fc191dec0cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:16.309563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aba1a2-67e8-4b83-8b4e-fc191dec0cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:16.331203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-45aba1a2-67e8-4b83-8b4e-fc191dec0cc5 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:16.332418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-45aba1a2-67e8-4b83-8b4e-fc191dec0cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:16.344255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1403/2848] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:16.348567 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e397df4-92fa-46d2-bfbf-303de62da97d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:16.358145 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e397df4-92fa-46d2-bfbf-303de62da97d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:16.363208 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00b300-a752-4e9a-8c99-2662de12c654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:16.363795 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00b300-a752-4e9a-8c99-2662de12c654 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:16.364069 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00b300-a752-4e9a-8c99-2662de12c654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:16.364340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00b300-a752-4e9a-8c99-2662de12c654 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:16.365004 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00b300-a752-4e9a-8c99-2662de12c654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:48:16.365303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00b300-a752-4e9a-8c99-2662de12c654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:16.366931 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00b300-a752-4e9a-8c99-2662de12c654 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:16.378657 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1446/2849] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:48:16.382709 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aba1a2-67e8-4b83-8b4e-fc191dec0cc5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:16.392696 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aba1a2-67e8-4b83-8b4e-fc191dec0cc5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:16.397382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b37e55-977b-468b-80f8-e5318a7ac232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:16.398010 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b37e55-977b-468b-80f8-e5318a7ac232 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:16.398290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b37e55-977b-468b-80f8-e5318a7ac232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:16.398562 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b37e55-977b-468b-80f8-e5318a7ac232 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:16.399181 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b37e55-977b-468b-80f8-e5318a7ac232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:16.401084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b37e55-977b-468b-80f8-e5318a7ac232 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:16.487852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1404/2850] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:48:16 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:16.491678 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00b300-a752-4e9a-8c99-2662de12c654 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:16.502141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00b300-a752-4e9a-8c99-2662de12c654 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:16.559028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8849293-53b5-4835-9eb5-ed1c5c1aa1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:16.559633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8849293-53b5-4835-9eb5-ed1c5c1aa1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:16.573367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8849293-53b5-4835-9eb5-ed1c5c1aa1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:16.573367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8849293-53b5-4835-9eb5-ed1c5c1aa1c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:16.573367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8849293-53b5-4835-9eb5-ed1c5c1aa1c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:16.573367 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8849293-53b5-4835-9eb5-ed1c5c1aa1c1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:16.801400 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1447/2851] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:48:16 2022] POST /identity/v3/users => generated 289 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:48:16.805743 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b37e55-977b-468b-80f8-e5318a7ac232 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:16.817323 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b37e55-977b-468b-80f8-e5318a7ac232 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:16.822148 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b09d74-85be-47f7-8e6c-17d84eee5d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:16.822755 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09d74-85be-47f7-8e6c-17d84eee5d5e admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:16.823030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09d74-85be-47f7-8e6c-17d84eee5d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:16.823380 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09d74-85be-47f7-8e6c-17d84eee5d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:16.824016 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b09d74-85be-47f7-8e6c-17d84eee5d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:16.825639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b09d74-85be-47f7-8e6c-17d84eee5d5e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:16.836795 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1405/2852] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:16.840748 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8849293-53b5-4835-9eb5-ed1c5c1aa1c1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:16.851015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8849293-53b5-4835-9eb5-ed1c5c1aa1c1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:16.855612 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c613f2-244f-48d9-8c3d-609b4de99c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:16.856207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c613f2-244f-48d9-8c3d-609b4de99c93 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:16.856484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c613f2-244f-48d9-8c3d-609b4de99c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:16.856851 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c613f2-244f-48d9-8c3d-609b4de99c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:16.857397 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c613f2-244f-48d9-8c3d-609b4de99c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:16.859036 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c613f2-244f-48d9-8c3d-609b4de99c93 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:16.870674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1448/2853] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:48:16.874812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b09d74-85be-47f7-8e6c-17d84eee5d5e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:16.886086 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b09d74-85be-47f7-8e6c-17d84eee5d5e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:16.890916 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a35087-aaf5-40a8-ae5c-dc3683bc625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:16.892103 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a35087-aaf5-40a8-ae5c-dc3683bc625d admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:16.892375 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a35087-aaf5-40a8-ae5c-dc3683bc625d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:16.892640 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a35087-aaf5-40a8-ae5c-dc3683bc625d admin admin] PATH_INFO: `/v3/projects/6cd002e767554f349692a4ca148c0a80/users/c12b7de7c6334714b857bb1ab7e779f3/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:16.901226 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:16.907634 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a35087-aaf5-40a8-ae5c-dc3683bc625d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd002e767554f349692a4ca148c0a80, user_id=c12b7de7c6334714b857bb1ab7e779f3, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:16.911860 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a35087-aaf5-40a8-ae5c-dc3683bc625d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:17.000434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1406/2854] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:16 2022] PUT /identity/v3/projects/6cd002e767554f349692a4ca148c0a80/users/c12b7de7c6334714b857bb1ab7e779f3/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:17.010908 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7911f-7cd6-475b-9b5e-2735df46cfbd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:17.016882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7911f-7cd6-475b-9b5e-2735df46cfbd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:17.016882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7911f-7cd6-475b-9b5e-2735df46cfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:17.039881 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-70d7911f-7cd6-475b-9b5e-2735df46cfbd None None] MFA Rules not processed for user `c12b7de7c6334714b857bb1ab7e779f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:17.041080 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-70d7911f-7cd6-475b-9b5e-2735df46cfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:17.104229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1449/2855] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:17.108936 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a35087-aaf5-40a8-ae5c-dc3683bc625d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:17.172049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a35087-aaf5-40a8-ae5c-dc3683bc625d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:17.177560 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa58fa8-eaff-49a0-a5a5-099db2e35155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:17.178155 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa58fa8-eaff-49a0-a5a5-099db2e35155 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:17.178434 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa58fa8-eaff-49a0-a5a5-099db2e35155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:17.178704 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa58fa8-eaff-49a0-a5a5-099db2e35155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:17.179333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa58fa8-eaff-49a0-a5a5-099db2e35155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:17.181228 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa58fa8-eaff-49a0-a5a5-099db2e35155 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:17.264597 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1407/2856] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:48:17 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:17.269091 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7911f-7cd6-475b-9b5e-2735df46cfbd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:17.274620 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:17.280474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7911f-7cd6-475b-9b5e-2735df46cfbd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:17.340359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11030b6b-c7e6-45bf-8778-0245670be593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:17.340964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11030b6b-c7e6-45bf-8778-0245670be593 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:17.341239 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11030b6b-c7e6-45bf-8778-0245670be593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:17.341543 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11030b6b-c7e6-45bf-8778-0245670be593 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:17.342163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11030b6b-c7e6-45bf-8778-0245670be593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:17.344057 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11030b6b-c7e6-45bf-8778-0245670be593 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:17.434603 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1450/2857] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:48:17 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:17.439370 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa58fa8-eaff-49a0-a5a5-099db2e35155 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:17.450408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa58fa8-eaff-49a0-a5a5-099db2e35155 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:17.455921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93b523-5fde-4109-9054-b98d30d0ad6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:17.456522 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93b523-5fde-4109-9054-b98d30d0ad6c admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:17.456826 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93b523-5fde-4109-9054-b98d30d0ad6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:17.457095 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93b523-5fde-4109-9054-b98d30d0ad6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:17.457769 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93b523-5fde-4109-9054-b98d30d0ad6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:17.459357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93b523-5fde-4109-9054-b98d30d0ad6c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:17.470266 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1408/2858] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:17.474587 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11030b6b-c7e6-45bf-8778-0245670be593 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:17.484505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11030b6b-c7e6-45bf-8778-0245670be593 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:17.489306 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef80f3a-bdcf-491b-98a4-e31cea36b612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:17.489932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef80f3a-bdcf-491b-98a4-e31cea36b612 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:17.490211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef80f3a-bdcf-491b-98a4-e31cea36b612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:17.490477 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef80f3a-bdcf-491b-98a4-e31cea36b612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:17.491111 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef80f3a-bdcf-491b-98a4-e31cea36b612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:17.492716 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef80f3a-bdcf-491b-98a4-e31cea36b612 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:17.502957 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1451/2859] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:48:17.507272 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93b523-5fde-4109-9054-b98d30d0ad6c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:17.517785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93b523-5fde-4109-9054-b98d30d0ad6c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:17.522340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a9b92-153e-4262-9a1e-c749c66b3bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:17.522969 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a9b92-153e-4262-9a1e-c749c66b3bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:17.523245 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a9b92-153e-4262-9a1e-c749c66b3bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:17.523514 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a9b92-153e-4262-9a1e-c749c66b3bd0 admin admin] PATH_INFO: `/v3/projects/c9cf8ab54c3e4594b063872726a5d68c/users/b3da614bb7b948d4b13a7f1817acbef3/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:17.537872 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a9b92-153e-4262-9a1e-c749c66b3bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9cf8ab54c3e4594b063872726a5d68c, user_id=b3da614bb7b948d4b13a7f1817acbef3, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:17.540517 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a9b92-153e-4262-9a1e-c749c66b3bd0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:17.612361 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1409/2860] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:17 2022] PUT /identity/v3/projects/c9cf8ab54c3e4594b063872726a5d68c/users/b3da614bb7b948d4b13a7f1817acbef3/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:17.623779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e464c0-696f-40ca-9207-32194718ce9e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:17.624066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e464c0-696f-40ca-9207-32194718ce9e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:17.624394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e464c0-696f-40ca-9207-32194718ce9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:17.653878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b4e464c0-696f-40ca-9207-32194718ce9e None None] MFA Rules not processed for user `b3da614bb7b948d4b13a7f1817acbef3`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:17.655028 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b4e464c0-696f-40ca-9207-32194718ce9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:17.718413 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1452/2861] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:17.723074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a9b92-153e-4262-9a1e-c749c66b3bd0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:17.786134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a9b92-153e-4262-9a1e-c749c66b3bd0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:17.790761 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de3c54-155b-4c0a-98ec-a75190e0cf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:17.791355 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de3c54-155b-4c0a-98ec-a75190e0cf4c admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:17.791630 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de3c54-155b-4c0a-98ec-a75190e0cf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:17.791901 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de3c54-155b-4c0a-98ec-a75190e0cf4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:17.792532 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de3c54-155b-4c0a-98ec-a75190e0cf4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:17.794480 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de3c54-155b-4c0a-98ec-a75190e0cf4c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:17.899650 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1410/2862] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:48:17 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:17.903945 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e464c0-696f-40ca-9207-32194718ce9e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:17.915177 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e464c0-696f-40ca-9207-32194718ce9e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:17.971594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d122f1-6751-4269-9a56-d1193c3247a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:17.972191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d122f1-6751-4269-9a56-d1193c3247a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:17.972474 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d122f1-6751-4269-9a56-d1193c3247a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:17.972750 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d122f1-6751-4269-9a56-d1193c3247a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:17.973385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d122f1-6751-4269-9a56-d1193c3247a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:17.975323 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d122f1-6751-4269-9a56-d1193c3247a2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:18.041531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:18.046572 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1453/2863] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:48:17 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:18.051262 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de3c54-155b-4c0a-98ec-a75190e0cf4c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:18.062094 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de3c54-155b-4c0a-98ec-a75190e0cf4c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:18.067531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fb70a-e344-4202-b869-a844cf3f87d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:18.068151 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fb70a-e344-4202-b869-a844cf3f87d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:18.068419 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fb70a-e344-4202-b869-a844cf3f87d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:18.068698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fb70a-e344-4202-b869-a844cf3f87d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:18.069339 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fb70a-e344-4202-b869-a844cf3f87d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:18.071330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fb70a-e344-4202-b869-a844cf3f87d4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:18.082078 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1411/2864] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:18.086108 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d122f1-6751-4269-9a56-d1193c3247a2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:18.096588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d122f1-6751-4269-9a56-d1193c3247a2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:18.101107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00225833-da44-4218-bbc5-ce57e1a789d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:18.101807 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00225833-da44-4218-bbc5-ce57e1a789d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:18.102070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00225833-da44-4218-bbc5-ce57e1a789d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:18.102343 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00225833-da44-4218-bbc5-ce57e1a789d3 admin admin] PATH_INFO: `/v3/projects/9faabcd10e3f4b579f2835809fff679e/users/63c2fce3314644b7ae23a74f7adda777/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:18.117381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00225833-da44-4218-bbc5-ce57e1a789d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9faabcd10e3f4b579f2835809fff679e, user_id=63c2fce3314644b7ae23a74f7adda777, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:18.120038 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00225833-da44-4218-bbc5-ce57e1a789d3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:18.178300 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1454/2865] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:18 2022] PUT /identity/v3/projects/9faabcd10e3f4b579f2835809fff679e/users/63c2fce3314644b7ae23a74f7adda777/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:18.182129 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fb70a-e344-4202-b869-a844cf3f87d4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:18.193003 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fb70a-e344-4202-b869-a844cf3f87d4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:18.200979 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:18.258250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacd8ed-b610-4b39-a35c-b87df29bfb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:18.258852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacd8ed-b610-4b39-a35c-b87df29bfb80 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:18.259124 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacd8ed-b610-4b39-a35c-b87df29bfb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:18.259393 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacd8ed-b610-4b39-a35c-b87df29bfb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:18.260026 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bacd8ed-b610-4b39-a35c-b87df29bfb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:18.261647 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bacd8ed-b610-4b39-a35c-b87df29bfb80 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:18.271537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1412/2866] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:18.276110 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00225833-da44-4218-bbc5-ce57e1a789d3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:18.289342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00225833-da44-4218-bbc5-ce57e1a789d3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:18.294209 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29209a-ede8-46f9-b6de-a2b1f495a6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:18.295035 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29209a-ede8-46f9-b6de-a2b1f495a6e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:18.295403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29209a-ede8-46f9-b6de-a2b1f495a6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:18.295775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29209a-ede8-46f9-b6de-a2b1f495a6e8 admin admin] PATH_INFO: `/v3/projects/9faabcd10e3f4b579f2835809fff679e/users/63c2fce3314644b7ae23a74f7adda777/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:18.299633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29209a-ede8-46f9-b6de-a2b1f495a6e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9faabcd10e3f4b579f2835809fff679e, user_id=63c2fce3314644b7ae23a74f7adda777, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:18.302778 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29209a-ede8-46f9-b6de-a2b1f495a6e8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:18.314421 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1455/2867] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:18 2022] PUT /identity/v3/projects/9faabcd10e3f4b579f2835809fff679e/users/63c2fce3314644b7ae23a74f7adda777/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:18.325049 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cace6-1a23-4bf4-9e6f-d3c5271f32ba None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:18.325776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cace6-1a23-4bf4-9e6f-d3c5271f32ba None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:18.325776 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cace6-1a23-4bf4-9e6f-d3c5271f32ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:18.353859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-929cace6-1a23-4bf4-9e6f-d3c5271f32ba None None] MFA Rules not processed for user `63c2fce3314644b7ae23a74f7adda777`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:18.355075 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-929cace6-1a23-4bf4-9e6f-d3c5271f32ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:18.423679 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1413/2868] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:18.435089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29209a-ede8-46f9-b6de-a2b1f495a6e8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:18.436764 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5d29209a-ede8-46f9-b6de-a2b1f495a6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:18.493769 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29209a-ede8-46f9-b6de-a2b1f495a6e8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:18.497342 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d503fc-23a5-4a42-8754-1e87539bb739 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:18.497962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d503fc-23a5-4a42-8754-1e87539bb739 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:18.498240 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d503fc-23a5-4a42-8754-1e87539bb739 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:18.498513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d503fc-23a5-4a42-8754-1e87539bb739 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:18.554588 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d503fc-23a5-4a42-8754-1e87539bb739 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:18.556020 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d503fc-23a5-4a42-8754-1e87539bb739 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:18.565549 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1456/2869] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:20.364768 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cace6-1a23-4bf4-9e6f-d3c5271f32ba None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:20.366219 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-929cace6-1a23-4bf4-9e6f-d3c5271f32ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:20.378834 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cace6-1a23-4bf4-9e6f-d3c5271f32ba None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:20.382299 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdfa67-3419-4d20-8b23-e5c81d49afcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:20.382890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdfa67-3419-4d20-8b23-e5c81d49afcc service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:20.383164 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdfa67-3419-4d20-8b23-e5c81d49afcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:20.383435 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdfa67-3419-4d20-8b23-e5c81d49afcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:20.440022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bdfa67-3419-4d20-8b23-e5c81d49afcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:20.441450 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bdfa67-3419-4d20-8b23-e5c81d49afcc service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:20.450509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1414/2870] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:21.718308 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d503fc-23a5-4a42-8754-1e87539bb739 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:21.727290 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d503fc-23a5-4a42-8754-1e87539bb739 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:21.730943 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa13d1-18e1-475e-b81b-35d4248fba0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:21.731569 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa13d1-18e1-475e-b81b-35d4248fba0b service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:21.731844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa13d1-18e1-475e-b81b-35d4248fba0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:21.732112 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa13d1-18e1-475e-b81b-35d4248fba0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:21.742046 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa13d1-18e1-475e-b81b-35d4248fba0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:21.743455 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa13d1-18e1-475e-b81b-35d4248fba0b service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:21.753552 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1457/2871] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:26.211310 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdfa67-3419-4d20-8b23-e5c81d49afcc service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:26.245363 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:26.275703 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdfa67-3419-4d20-8b23-e5c81d49afcc service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:26.280749 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:26.281369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:26.281686 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:26.281956 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] PATH_INFO: `/v3/users/c12b7de7c6334714b857bb1ab7e779f3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:26.283096 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c12b7de7c6334714b857bb1ab7e779f3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:26.285055 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:26.465357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c12b7de7c6334714b857bb1ab7e779f3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:26.465951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c12b7de7c6334714b857bb1ab7e779f3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:26.520386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c12b7de7c6334714b857bb1ab7e779f3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:26.532427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c12b7de7c6334714b857bb1ab7e779f3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:26.540115 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c12b7de7c6334714b857bb1ab7e779f3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:26.549144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1415/2872] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:48:26 2022] DELETE /identity/v3/users/c12b7de7c6334714b857bb1ab7e779f3 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:26.556450 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2c594-d754-400a-be15-7b5a83c7cfc7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:26.557968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2c594-d754-400a-be15-7b5a83c7cfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:26.557968 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2c594-d754-400a-be15-7b5a83c7cfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:26.577223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-57f2c594-d754-400a-be15-7b5a83c7cfc7 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:26.578458 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-57f2c594-d754-400a-be15-7b5a83c7cfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:26.637763 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1458/2873] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:26.648255 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:26.649700 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:26.706706 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414a714-2fe3-4824-89c5-763a30206281 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:26.711203 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f5cef-837c-4454-adcf-05ea39e837b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:26.711752 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f5cef-837c-4454-adcf-05ea39e837b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:26.712036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f5cef-837c-4454-adcf-05ea39e837b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:26.712308 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f5cef-837c-4454-adcf-05ea39e837b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:26.721804 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f5cef-837c-4454-adcf-05ea39e837b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:26.723531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f5cef-837c-4454-adcf-05ea39e837b5 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:26.733347 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1416/2874] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:27.122182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2c594-d754-400a-be15-7b5a83c7cfc7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:27.123619 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-57f2c594-d754-400a-be15-7b5a83c7cfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:27.132978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:27.137622 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2c594-d754-400a-be15-7b5a83c7cfc7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:27.142303 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:27.143034 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:27.143385 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:27.143725 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] PATH_INFO: `/v3/projects/6cd002e767554f349692a4ca148c0a80` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:27.145736 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cd002e767554f349692a4ca148c0a80)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:27.147864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:27.235039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cd002e767554f349692a4ca148c0a80', 'request_id': 'req-23cea94b-385f-411a-b126-cada01b01b85'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:27.291416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cd002e767554f349692a4ca148c0a80', 'request_id': 'req-23cea94b-385f-411a-b126-cada01b01b85'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:27.297889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cd002e767554f349692a4ca148c0a80', 'request_id': 'req-23cea94b-385f-411a-b126-cada01b01b85'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:27.308228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] The token cache is being invalidate because project 6cd002e767554f349692a4ca148c0a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:48:27.308509 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:27.309139 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:27.310640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1459/2875] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:48:27 2022] DELETE /identity/v3/projects/6cd002e767554f349692a4ca148c0a80 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:27.314594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f5cef-837c-4454-adcf-05ea39e837b5 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:27.325282 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f5cef-837c-4454-adcf-05ea39e837b5 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:27.375297 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:27.384859 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:27.385508 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:27.385792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:27.386066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] PATH_INFO: `/v3/users/b3da614bb7b948d4b13a7f1817acbef3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:27.387185 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3da614bb7b948d4b13a7f1817acbef3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:27.389188 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:27.579906 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3da614bb7b948d4b13a7f1817acbef3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:27.580475 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3da614bb7b948d4b13a7f1817acbef3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:27.662918 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3da614bb7b948d4b13a7f1817acbef3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:27.675681 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3da614bb7b948d4b13a7f1817acbef3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:27.683429 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3da614bb7b948d4b13a7f1817acbef3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:27.689279 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1417/2876] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:48:27 2022] DELETE /identity/v3/users/b3da614bb7b948d4b13a7f1817acbef3 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:28.098933 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:28.101389 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:28.115427 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cea94b-385f-411a-b126-cada01b01b85 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:28.174720 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:28.175341 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:28.175617 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:28.175912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] PATH_INFO: `/v3/projects/c9cf8ab54c3e4594b063872726a5d68c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:28.177304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9cf8ab54c3e4594b063872726a5d68c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:28.179231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:28.274665 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9cf8ab54c3e4594b063872726a5d68c', 'request_id': 'req-3de73ed3-2928-47ea-9b67-6ae33166f62b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:28.365947 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9cf8ab54c3e4594b063872726a5d68c', 'request_id': 'req-3de73ed3-2928-47ea-9b67-6ae33166f62b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:28.372576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9cf8ab54c3e4594b063872726a5d68c', 'request_id': 'req-3de73ed3-2928-47ea-9b67-6ae33166f62b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:28.383575 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] The token cache is being invalidate because project c9cf8ab54c3e4594b063872726a5d68c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:48:28.383864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:28.384505 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:28.386038 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1460/2877] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:48:28 2022] DELETE /identity/v3/projects/c9cf8ab54c3e4594b063872726a5d68c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:28.390032 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:28.391463 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:28.404293 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f56079-0934-411f-a830-04a306c175c3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:28.461476 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:28.462146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:28.462422 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:28.462689 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] PATH_INFO: `/v3/users/63c2fce3314644b7ae23a74f7adda777` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:28.463785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c2fce3314644b7ae23a74f7adda777)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:28.465732 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:28.643462 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c2fce3314644b7ae23a74f7adda777'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:28.644039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c2fce3314644b7ae23a74f7adda777'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:28.720301 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c2fce3314644b7ae23a74f7adda777'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:28.733483 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c2fce3314644b7ae23a74f7adda777'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:28.735687 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:28.741371 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4168a35a-0b8c-4f4c-92e9-403b85007d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c2fce3314644b7ae23a74f7adda777'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:28.747289 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1418/2878] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:48:28 2022] DELETE /identity/v3/users/63c2fce3314644b7ae23a74f7adda777 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:29.134475 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:29.135974 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:29.151179 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de73ed3-2928-47ea-9b67-6ae33166f62b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:29.163212 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:29.208289 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:29.208949 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:29.209229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:29.209533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] PATH_INFO: `/v3/projects/9faabcd10e3f4b579f2835809fff679e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:29.210770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9faabcd10e3f4b579f2835809fff679e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:29.212666 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:29.354318 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9faabcd10e3f4b579f2835809fff679e', 'request_id': 'req-7525a9da-edd4-4c3b-8916-dd157fe7ac30'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:29.428220 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9faabcd10e3f4b579f2835809fff679e', 'request_id': 'req-7525a9da-edd4-4c3b-8916-dd157fe7ac30'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:29.434640 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9faabcd10e3f4b579f2835809fff679e', 'request_id': 'req-7525a9da-edd4-4c3b-8916-dd157fe7ac30'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:29.445600 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] The token cache is being invalidate because project 9faabcd10e3f4b579f2835809fff679e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:48:29.445888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:29.446514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7525a9da-edd4-4c3b-8916-dd157fe7ac30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:29.448097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1461/2879] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:48:29 2022] DELETE /identity/v3/projects/9faabcd10e3f4b579f2835809fff679e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:29.453022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595cdac-1140-4427-9692-91c6049afe1e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:29.453515 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595cdac-1140-4427-9692-91c6049afe1e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:29.453707 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595cdac-1140-4427-9692-91c6049afe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:29.474313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c595cdac-1140-4427-9692-91c6049afe1e None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:29.475538 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c595cdac-1140-4427-9692-91c6049afe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:29.532406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1419/2880] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:29.541089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440407f-d066-4a62-944e-e5194c72d816 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:29.541381 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440407f-d066-4a62-944e-e5194c72d816 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:29.541694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440407f-d066-4a62-944e-e5194c72d816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:29.561286 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b440407f-d066-4a62-944e-e5194c72d816 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:29.562601 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b440407f-d066-4a62-944e-e5194c72d816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:29.571811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1462/2881] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:29.575921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595cdac-1140-4427-9692-91c6049afe1e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:29.586340 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595cdac-1140-4427-9692-91c6049afe1e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:29.590940 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4559013-c0c2-4d2f-b39c-fba5658dd187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:29.591533 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4559013-c0c2-4d2f-b39c-fba5658dd187 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:29.591812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4559013-c0c2-4d2f-b39c-fba5658dd187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:29.592084 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4559013-c0c2-4d2f-b39c-fba5658dd187 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:29.592725 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4559013-c0c2-4d2f-b39c-fba5658dd187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:48:29.593029 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4559013-c0c2-4d2f-b39c-fba5658dd187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:29.594651 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4559013-c0c2-4d2f-b39c-fba5658dd187 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:29.606445 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1420/2882] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:48:29.610582 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440407f-d066-4a62-944e-e5194c72d816 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:29.619850 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440407f-d066-4a62-944e-e5194c72d816 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:29.624676 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4cb14-e079-4ab0-bb7c-8cd2df319707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:29.625269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4cb14-e079-4ab0-bb7c-8cd2df319707 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:29.625590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4cb14-e079-4ab0-bb7c-8cd2df319707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:29.625863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4cb14-e079-4ab0-bb7c-8cd2df319707 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:29.626479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4cb14-e079-4ab0-bb7c-8cd2df319707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:29.628394 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4cb14-e079-4ab0-bb7c-8cd2df319707 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:29.710556 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1463/2883] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:48:29 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:48:29.714805 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4559013-c0c2-4d2f-b39c-fba5658dd187 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:29.725554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4559013-c0c2-4d2f-b39c-fba5658dd187 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:29.782792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9fd87-18fc-4816-a62a-bd9b4a4dc43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:29.783391 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9fd87-18fc-4816-a62a-bd9b4a4dc43e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:29.783665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9fd87-18fc-4816-a62a-bd9b4a4dc43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:29.783993 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9fd87-18fc-4816-a62a-bd9b4a4dc43e admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:29.784627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9fd87-18fc-4816-a62a-bd9b4a4dc43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:29.786981 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9fd87-18fc-4816-a62a-bd9b4a4dc43e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:29.863792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1421/2884] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:48:29 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:48:29.868174 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4cb14-e079-4ab0-bb7c-8cd2df319707 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:29.878387 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4cb14-e079-4ab0-bb7c-8cd2df319707 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:29.883873 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2879d2-d79e-4ab6-932e-515bdb927388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:29.884463 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2879d2-d79e-4ab6-932e-515bdb927388 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:29.884742 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2879d2-d79e-4ab6-932e-515bdb927388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:29.885009 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2879d2-d79e-4ab6-932e-515bdb927388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:29.885682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2879d2-d79e-4ab6-932e-515bdb927388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:29.887260 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2879d2-d79e-4ab6-932e-515bdb927388 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:29.897401 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1464/2885] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:29.901478 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9fd87-18fc-4816-a62a-bd9b4a4dc43e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:29.911699 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9fd87-18fc-4816-a62a-bd9b4a4dc43e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:29.916296 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83775aeb-be9e-4df2-ac30-bd67a91671c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:29.918691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83775aeb-be9e-4df2-ac30-bd67a91671c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:29.918691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83775aeb-be9e-4df2-ac30-bd67a91671c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:29.918691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83775aeb-be9e-4df2-ac30-bd67a91671c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:29.918691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83775aeb-be9e-4df2-ac30-bd67a91671c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:29.919616 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83775aeb-be9e-4df2-ac30-bd67a91671c4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:29.930872 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1422/2886] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:48:29.934878 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2879d2-d79e-4ab6-932e-515bdb927388 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:29.946163 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2879d2-d79e-4ab6-932e-515bdb927388 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:29.950651 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be874a4b-36ca-4018-822b-e58a308d8922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:29.951291 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be874a4b-36ca-4018-822b-e58a308d8922 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:29.951566 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be874a4b-36ca-4018-822b-e58a308d8922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:29.951839 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be874a4b-36ca-4018-822b-e58a308d8922 admin admin] PATH_INFO: `/v3/projects/2f22ba19901d489db974b2f09cee6e0b/users/46ecccda1e7b49969b26c6414c9c490a/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:29.966899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be874a4b-36ca-4018-822b-e58a308d8922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f22ba19901d489db974b2f09cee6e0b, user_id=46ecccda1e7b49969b26c6414c9c490a, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:29.969533 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be874a4b-36ca-4018-822b-e58a308d8922 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:30.040847 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1465/2887] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:48:29 2022] PUT /identity/v3/projects/2f22ba19901d489db974b2f09cee6e0b/users/46ecccda1e7b49969b26c6414c9c490a/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:30.046006 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6841-72b3-473d-ad12-c4d7a698e40a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:30.046284 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6841-72b3-473d-ad12-c4d7a698e40a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:30.046554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6841-72b3-473d-ad12-c4d7a698e40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:30.065748 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a3bf6841-72b3-473d-ad12-c4d7a698e40a None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:30.067287 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a3bf6841-72b3-473d-ad12-c4d7a698e40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:30.117734 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:30.126713 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1423/2888] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:48:30.139728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be874a4b-36ca-4018-822b-e58a308d8922 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:30.141480 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-be874a4b-36ca-4018-822b-e58a308d8922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:30.188305 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:30.199857 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be874a4b-36ca-4018-822b-e58a308d8922 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:30.203576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7c181-63e9-4fd5-b147-fe4043a9074b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:30.204171 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7c181-63e9-4fd5-b147-fe4043a9074b service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:30.204448 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7c181-63e9-4fd5-b147-fe4043a9074b service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:30.204784 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7c181-63e9-4fd5-b147-fe4043a9074b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:30.214715 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7c181-63e9-4fd5-b147-fe4043a9074b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:30.216136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7c181-63e9-4fd5-b147-fe4043a9074b service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:30.226304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1466/2889] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:31.499039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b4e9a-929f-40c6-813f-0f6c15c8be59 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:31.503525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b4e9a-929f-40c6-813f-0f6c15c8be59 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:31.503525 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b4e9a-929f-40c6-813f-0f6c15c8be59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:31.529568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a28b4e9a-929f-40c6-813f-0f6c15c8be59 None None] MFA Rules not processed for user `46ecccda1e7b49969b26c6414c9c490a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:31.530796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a28b4e9a-929f-40c6-813f-0f6c15c8be59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:31.598575 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1424/2890] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:31.609062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7c181-63e9-4fd5-b147-fe4043a9074b service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:31.610512 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8ba7c181-63e9-4fd5-b147-fe4043a9074b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:31.667641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7c181-63e9-4fd5-b147-fe4043a9074b service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:31.671244 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb67ba-6773-4c49-ae5c-3c6e9f2b4f53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:31.671845 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb67ba-6773-4c49-ae5c-3c6e9f2b4f53 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:31.672128 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb67ba-6773-4c49-ae5c-3c6e9f2b4f53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:31.672403 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb67ba-6773-4c49-ae5c-3c6e9f2b4f53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:31.680797 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb67ba-6773-4c49-ae5c-3c6e9f2b4f53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:31.682266 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb67ba-6773-4c49-ae5c-3c6e9f2b4f53 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:31.690997 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1467/2891] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:33.512418 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4d88-8aa6-48cb-83d2-84bf51b4e382 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:33.512921 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4d88-8aa6-48cb-83d2-84bf51b4e382 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:33.513061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4d88-8aa6-48cb-83d2-84bf51b4e382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:33.533957 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f8cf4d88-8aa6-48cb-83d2-84bf51b4e382 None None] MFA Rules not processed for user `46ecccda1e7b49969b26c6414c9c490a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:33.535137 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f8cf4d88-8aa6-48cb-83d2-84bf51b4e382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:33.542925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1425/2892] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:33.794134 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb67ba-6773-4c49-ae5c-3c6e9f2b4f53 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:33.803775 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb67ba-6773-4c49-ae5c-3c6e9f2b4f53 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:33.807353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fed157-2a48-46fc-972b-81569a63704e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:33.807946 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fed157-2a48-46fc-972b-81569a63704e service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:33.808227 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fed157-2a48-46fc-972b-81569a63704e service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:33.808507 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fed157-2a48-46fc-972b-81569a63704e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:33.810355 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a5fed157-2a48-46fc-972b-81569a63704e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:33.820978 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fed157-2a48-46fc-972b-81569a63704e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:33.822508 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fed157-2a48-46fc-972b-81569a63704e service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:33.830608 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1468/2893] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:34.490424 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf4d88-8aa6-48cb-83d2-84bf51b4e382 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:34.491820 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f8cf4d88-8aa6-48cb-83d2-84bf51b4e382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:34.564594 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf4d88-8aa6-48cb-83d2-84bf51b4e382 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:34.570675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba19e1-915a-4c24-870d-b2122e3a0b7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:34.571319 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba19e1-915a-4c24-870d-b2122e3a0b7f service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:34.571602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba19e1-915a-4c24-870d-b2122e3a0b7f service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:34.571877 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba19e1-915a-4c24-870d-b2122e3a0b7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:34.573675 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-85ba19e1-915a-4c24-870d-b2122e3a0b7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:34.585555 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba19e1-915a-4c24-870d-b2122e3a0b7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:34.587138 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba19e1-915a-4c24-870d-b2122e3a0b7f service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:34.604004 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1426/2894] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:37.593070 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fed157-2a48-46fc-972b-81569a63704e service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:37.602770 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fed157-2a48-46fc-972b-81569a63704e service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:37.606487 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f81672-00d8-4dde-891e-f52eb8f38c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:37.612211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f81672-00d8-4dde-891e-f52eb8f38c14 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:37.612211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f81672-00d8-4dde-891e-f52eb8f38c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:37.612211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f81672-00d8-4dde-891e-f52eb8f38c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:37.612211 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b8f81672-00d8-4dde-891e-f52eb8f38c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:37.620073 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f81672-00d8-4dde-891e-f52eb8f38c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:37.621524 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f81672-00d8-4dde-891e-f52eb8f38c14 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:37.630147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1469/2895] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:38.397595 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba19e1-915a-4c24-870d-b2122e3a0b7f service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:38.399305 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-85ba19e1-915a-4c24-870d-b2122e3a0b7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:38.483619 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba19e1-915a-4c24-870d-b2122e3a0b7f service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:38.492531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4e9d9-9bfb-44cb-a112-4a83ea85f53b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:38.493149 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4e9d9-9bfb-44cb-a112-4a83ea85f53b service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:38.493430 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4e9d9-9bfb-44cb-a112-4a83ea85f53b service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:38.493737 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4e9d9-9bfb-44cb-a112-4a83ea85f53b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:38.495538 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a2a4e9d9-9bfb-44cb-a112-4a83ea85f53b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:38.509631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4e9d9-9bfb-44cb-a112-4a83ea85f53b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:38.510583 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4e9d9-9bfb-44cb-a112-4a83ea85f53b service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:38.516204 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:38.525089 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1427/2896] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:48:43.201762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f81672-00d8-4dde-891e-f52eb8f38c14 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:43.203207 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b8f81672-00d8-4dde-891e-f52eb8f38c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:43.220363 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f81672-00d8-4dde-891e-f52eb8f38c14 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:43.225959 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf44c58-c504-4bb8-97ca-4e5967b8b2b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:43.226555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf44c58-c504-4bb8-97ca-4e5967b8b2b9 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:43.226831 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf44c58-c504-4bb8-97ca-4e5967b8b2b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:43.227097 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf44c58-c504-4bb8-97ca-4e5967b8b2b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:43.228759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cdf44c58-c504-4bb8-97ca-4e5967b8b2b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:43.241422 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf44c58-c504-4bb8-97ca-4e5967b8b2b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:43.243164 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf44c58-c504-4bb8-97ca-4e5967b8b2b9 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:43.255169 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1470/2897] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:44.049120 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4e9d9-9bfb-44cb-a112-4a83ea85f53b service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:44.058785 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4e9d9-9bfb-44cb-a112-4a83ea85f53b service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:44.062441 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f45e37-69a0-448e-9e31-f4cbee6375f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:44.063036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f45e37-69a0-448e-9e31-f4cbee6375f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:44.063313 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f45e37-69a0-448e-9e31-f4cbee6375f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:44.063582 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f45e37-69a0-448e-9e31-f4cbee6375f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:44.065316 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-04f45e37-69a0-448e-9e31-f4cbee6375f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:44.124816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f45e37-69a0-448e-9e31-f4cbee6375f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:44.126350 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f45e37-69a0-448e-9e31-f4cbee6375f1 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:44.135717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1428/2898] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:50.784780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf44c58-c504-4bb8-97ca-4e5967b8b2b9 service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:50.787010 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cdf44c58-c504-4bb8-97ca-4e5967b8b2b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:50.798412 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf44c58-c504-4bb8-97ca-4e5967b8b2b9 service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:50.802121 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0998539-66b0-46c8-826f-5cd3b6774bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:50.802791 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0998539-66b0-46c8-826f-5cd3b6774bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:50.803089 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0998539-66b0-46c8-826f-5cd3b6774bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:50.803339 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0998539-66b0-46c8-826f-5cd3b6774bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:50.805022 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a0998539-66b0-46c8-826f-5cd3b6774bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:50.816590 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0998539-66b0-46c8-826f-5cd3b6774bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:50.818465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0998539-66b0-46c8-826f-5cd3b6774bc7 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:50.828614 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1471/2899] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:52.101234 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e298e-cc9e-4b74-8363-09c05ca69854 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:52.101809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e298e-cc9e-4b74-8363-09c05ca69854 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:52.101809 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e298e-cc9e-4b74-8363-09c05ca69854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:52.121101 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1a9e298e-cc9e-4b74-8363-09c05ca69854 None None] MFA Rules not processed for user `46ecccda1e7b49969b26c6414c9c490a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:48:52.122329 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1a9e298e-cc9e-4b74-8363-09c05ca69854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:52.129320 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1429/2900] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:48:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:48:54.415114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0998539-66b0-46c8-826f-5cd3b6774bc7 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:54.416779 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a0998539-66b0-46c8-826f-5cd3b6774bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:54.435063 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0998539-66b0-46c8-826f-5cd3b6774bc7 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:54.440147 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e557b-d292-4c68-85ac-ff8773861e15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:54.441899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e557b-d292-4c68-85ac-ff8773861e15 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:54.441899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e557b-d292-4c68-85ac-ff8773861e15 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:54.441899 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e557b-d292-4c68-85ac-ff8773861e15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:54.444083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-825e557b-d292-4c68-85ac-ff8773861e15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:54.458141 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e557b-d292-4c68-85ac-ff8773861e15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:54.460228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e557b-d292-4c68-85ac-ff8773861e15 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:54.470916 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1472/2901] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:48:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:59.173030 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e298e-cc9e-4b74-8363-09c05ca69854 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:59.174520 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1a9e298e-cc9e-4b74-8363-09c05ca69854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:48:59.188309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e298e-cc9e-4b74-8363-09c05ca69854 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:59.194074 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:59.194701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:59.195045 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:59.195659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] PATH_INFO: `/v3/users/45e1b0901d174b10b49709d74d5fe5d6` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:59.196962 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e1b0901d174b10b49709d74d5fe5d6)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:59.199017 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:59.373333 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e1b0901d174b10b49709d74d5fe5d6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:59.374130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e1b0901d174b10b49709d74d5fe5d6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:59.474309 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e1b0901d174b10b49709d74d5fe5d6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:59.486598 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e1b0901d174b10b49709d74d5fe5d6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:59.493536 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e1b0901d174b10b49709d74d5fe5d6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:48:59.499379 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1430/2902] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:48:59 2022] DELETE /identity/v3/users/45e1b0901d174b10b49709d74d5fe5d6 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:48:59.772754 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e557b-d292-4c68-85ac-ff8773861e15 service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:48:59.784419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e557b-d292-4c68-85ac-ff8773861e15 service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:48:59.848882 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:48:59.849641 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:48:59.849953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:48:59.850304 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] PATH_INFO: `/v3/projects/138f37da900443bea3ab358845a7c773` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:48:59.851431 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=138f37da900443bea3ab358845a7c773)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:48:59.853316 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:48:59.932673 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:48:59.950252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '138f37da900443bea3ab358845a7c773', 'request_id': 'req-2f819f11-4a3a-46cd-b654-1099333cd5fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.016479 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '138f37da900443bea3ab358845a7c773', 'request_id': 'req-2f819f11-4a3a-46cd-b654-1099333cd5fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.023484 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '138f37da900443bea3ab358845a7c773', 'request_id': 'req-2f819f11-4a3a-46cd-b654-1099333cd5fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.034555 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] The token cache is being invalidate because project 138f37da900443bea3ab358845a7c773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:49:00.034934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.035632 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.037844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1473/2903] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:48:59 2022] DELETE /identity/v3/projects/138f37da900443bea3ab358845a7c773 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:49:00.041941 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:00.043426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:00.057009 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24128dc-1b65-4dc1-a92a-4bfe6999bf7b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:00.116211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:00.116842 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:00.117118 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:00.117424 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] PATH_INFO: `/v3/users/830504c4475248dfa8b0bf0f18397c0a` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:00.118600 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=830504c4475248dfa8b0bf0f18397c0a)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:00.120531 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:00.122128 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:00.236554 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830504c4475248dfa8b0bf0f18397c0a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.237225 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830504c4475248dfa8b0bf0f18397c0a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.279295 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830504c4475248dfa8b0bf0f18397c0a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.292278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830504c4475248dfa8b0bf0f18397c0a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.299143 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-79b23a54-cb72-4dac-b48a-559a119e91d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830504c4475248dfa8b0bf0f18397c0a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.307535 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1431/2904] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:49:00 2022] DELETE /identity/v3/users/830504c4475248dfa8b0bf0f18397c0a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:49:00.520700 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:00.522932 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:00.537812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f819f11-4a3a-46cd-b654-1099333cd5fa admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:00.594833 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:00.595453 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:00.595730 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:00.595996 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] PATH_INFO: `/v3/projects/d9cc40feb4764538ba1ea556b8fb2b37` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:00.597090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9cc40feb4764538ba1ea556b8fb2b37)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:00.599001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:00.673762 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9cc40feb4764538ba1ea556b8fb2b37', 'request_id': 'req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.731195 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9cc40feb4764538ba1ea556b8fb2b37', 'request_id': 'req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.738107 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9cc40feb4764538ba1ea556b8fb2b37', 'request_id': 'req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.749083 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] The token cache is being invalidate because project d9cc40feb4764538ba1ea556b8fb2b37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:49:00.749374 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.750047 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9bbb8875-331f-49a5-a258-3f9f9b7e89b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:00.751525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1474/2905] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:49:00 2022] DELETE /identity/v3/projects/d9cc40feb4764538ba1ea556b8fb2b37 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:49:00.756611 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c06f-d3ec-47de-b7b7-68fda4c42107 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:00.756890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c06f-d3ec-47de-b7b7-68fda4c42107 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:00.757166 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c06f-d3ec-47de-b7b7-68fda4c42107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:00.777169 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-83a0c06f-d3ec-47de-b7b7-68fda4c42107 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:49:00.778406 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-83a0c06f-d3ec-47de-b7b7-68fda4c42107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:00.839493 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1432/2906] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:49:00.847789 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23439bf9-7ba7-4c7f-a869-4f79624e9770 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:00.848301 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23439bf9-7ba7-4c7f-a869-4f79624e9770 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:00.848426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23439bf9-7ba7-4c7f-a869-4f79624e9770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:00.869674 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-23439bf9-7ba7-4c7f-a869-4f79624e9770 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:49:00.870870 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-23439bf9-7ba7-4c7f-a869-4f79624e9770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:00.878728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1475/2907] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:49:00.883712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0c06f-d3ec-47de-b7b7-68fda4c42107 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:00.893779 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0c06f-d3ec-47de-b7b7-68fda4c42107 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:00.899714 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a73d32-b827-41cf-9fa6-e407e0db0bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:00.900300 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a73d32-b827-41cf-9fa6-e407e0db0bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:00.900576 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a73d32-b827-41cf-9fa6-e407e0db0bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:00.900887 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a73d32-b827-41cf-9fa6-e407e0db0bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:00.901567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a73d32-b827-41cf-9fa6-e407e0db0bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:49:00.901867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a73d32-b827-41cf-9fa6-e407e0db0bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:00.903463 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a73d32-b827-41cf-9fa6-e407e0db0bf4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:00.915796 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1433/2908] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 1 05:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:49:00.920066 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23439bf9-7ba7-4c7f-a869-4f79624e9770 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:00.929694 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23439bf9-7ba7-4c7f-a869-4f79624e9770 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:00.934461 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a61989-4130-4a69-8d8f-a461ee14f74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:00.935050 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a61989-4130-4a69-8d8f-a461ee14f74a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:00.935328 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a61989-4130-4a69-8d8f-a461ee14f74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:00.935599 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a61989-4130-4a69-8d8f-a461ee14f74a admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:00.936228 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a61989-4130-4a69-8d8f-a461ee14f74a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:00.941172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a61989-4130-4a69-8d8f-a461ee14f74a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:01.040889 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1476/2909] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:49:00 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:49:01.045119 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a73d32-b827-41cf-9fa6-e407e0db0bf4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:01.055530 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a73d32-b827-41cf-9fa6-e407e0db0bf4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:01.112373 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439a871-4716-4353-b6ce-52cc9d3785d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:01.112890 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a871-4716-4353-b6ce-52cc9d3785d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:01.113165 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a871-4716-4353-b6ce-52cc9d3785d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:01.113433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a871-4716-4353-b6ce-52cc9d3785d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:01.114087 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439a871-4716-4353-b6ce-52cc9d3785d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:01.116066 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439a871-4716-4353-b6ce-52cc9d3785d6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:01.126842 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:01.234903 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1434/2910] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:49:01 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:49:01.239265 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a61989-4130-4a69-8d8f-a461ee14f74a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:01.250774 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a61989-4130-4a69-8d8f-a461ee14f74a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:01.256109 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3693211-61a9-4d9a-b4e5-bd390950b1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:01.256866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693211-61a9-4d9a-b4e5-bd390950b1ab admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:01.257019 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693211-61a9-4d9a-b4e5-bd390950b1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:01.257247 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693211-61a9-4d9a-b4e5-bd390950b1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:01.257948 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3693211-61a9-4d9a-b4e5-bd390950b1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:01.261282 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3693211-61a9-4d9a-b4e5-bd390950b1ab admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:01.272249 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1477/2911] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:49:01.276321 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439a871-4716-4353-b6ce-52cc9d3785d6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:01.286808 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439a871-4716-4353-b6ce-52cc9d3785d6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:01.292824 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bd2c2-2063-4ffd-ae12-7a0d8e2bef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:01.293436 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd2c2-2063-4ffd-ae12-7a0d8e2bef49 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:01.293742 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd2c2-2063-4ffd-ae12-7a0d8e2bef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:01.294014 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd2c2-2063-4ffd-ae12-7a0d8e2bef49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:01.294667 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bd2c2-2063-4ffd-ae12-7a0d8e2bef49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:01.296944 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bd2c2-2063-4ffd-ae12-7a0d8e2bef49 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:01.312492 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1435/2912] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:49:01.313693 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3693211-61a9-4d9a-b4e5-bd390950b1ab admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:01.321697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:01.325629 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3693211-61a9-4d9a-b4e5-bd390950b1ab admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:01.334794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bf3a9-4537-4698-a051-80d9459dc019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:01.334794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf3a9-4537-4698-a051-80d9459dc019 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:01.334794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf3a9-4537-4698-a051-80d9459dc019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:01.334794 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf3a9-4537-4698-a051-80d9459dc019 admin admin] PATH_INFO: `/v3/projects/8929d2f4d729433bb807e9f0890c9b1f/users/99c0ab8ddd164b44a22370a7cc1042be/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:01.348078 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bf3a9-4537-4698-a051-80d9459dc019 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8929d2f4d729433bb807e9f0890c9b1f, user_id=99c0ab8ddd164b44a22370a7cc1042be, role_id=ef3d1cb6e6f04bbbb52f18e977f6fb75)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:01.350861 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bf3a9-4537-4698-a051-80d9459dc019 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:01.446426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1478/2913] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:49:01 2022] PUT /identity/v3/projects/8929d2f4d729433bb807e9f0890c9b1f/users/99c0ab8ddd164b44a22370a7cc1042be/roles/ef3d1cb6e6f04bbbb52f18e977f6fb75 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:49:01.451524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399a567-e901-47e4-8199-04c37e647d91 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:01.454698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399a567-e901-47e4-8199-04c37e647d91 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:01.454698 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399a567-e901-47e4-8199-04c37e647d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:01.472814 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8399a567-e901-47e4-8199-04c37e647d91 None None] MFA Rules not processed for user `9d4d38ef7ecd40a380aba27a2b564d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:49:01.474070 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8399a567-e901-47e4-8199-04c37e647d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:01.534054 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1436/2914] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:49:01.544155 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bf3a9-4537-4698-a051-80d9459dc019 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:01.545858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5d1bf3a9-4537-4698-a051-80d9459dc019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:01.602710 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bf3a9-4537-4698-a051-80d9459dc019 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:01.606275 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff624e-c685-43cb-9393-d76c3975e3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:01.606864 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff624e-c685-43cb-9393-d76c3975e3bc service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:01.607142 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff624e-c685-43cb-9393-d76c3975e3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:01.607416 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff624e-c685-43cb-9393-d76c3975e3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:01.617340 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff624e-c685-43cb-9393-d76c3975e3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:01.618777 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff624e-c685-43cb-9393-d76c3975e3bc service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:01.627912 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1479/2915] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:49:06.916792 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f34bc-0108-4464-a05f-9030528959b9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:06.917334 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f34bc-0108-4464-a05f-9030528959b9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:06.917477 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f34bc-0108-4464-a05f-9030528959b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:06.949065 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8e5f34bc-0108-4464-a05f-9030528959b9 None None] MFA Rules not processed for user `99c0ab8ddd164b44a22370a7cc1042be`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:49:06.950354 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8e5f34bc-0108-4464-a05f-9030528959b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:07.029881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1437/2916] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:49:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:49:07.034965 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff624e-c685-43cb-9393-d76c3975e3bc service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:07.045984 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff624e-c685-43cb-9393-d76c3975e3bc service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:07.052039 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf80d1-07a4-4694-9b90-7f13bdf25a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:07.052745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf80d1-07a4-4694-9b90-7f13bdf25a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:07.053086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf80d1-07a4-4694-9b90-7f13bdf25a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:07.053423 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf80d1-07a4-4694-9b90-7f13bdf25a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:07.054172 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf80d1-07a4-4694-9b90-7f13bdf25a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:07.056325 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf80d1-07a4-4694-9b90-7f13bdf25a96 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:07.139559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1480/2917] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 1 05:49:07 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:49:07.143756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f34bc-0108-4464-a05f-9030528959b9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:07.156357 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f34bc-0108-4464-a05f-9030528959b9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:07.216817 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfe83a-f260-4a9d-9274-d6c283ddc398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:07.217812 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfe83a-f260-4a9d-9274-d6c283ddc398 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:07.218091 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfe83a-f260-4a9d-9274-d6c283ddc398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:07.218386 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfe83a-f260-4a9d-9274-d6c283ddc398 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:07.219022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfe83a-f260-4a9d-9274-d6c283ddc398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:07.221011 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfe83a-f260-4a9d-9274-d6c283ddc398 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:07.313268 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1438/2918] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 1 05:49:07 2022] POST /identity/v3/users => generated 295 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:49:07.314780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf80d1-07a4-4694-9b90-7f13bdf25a96 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:07.329229 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf80d1-07a4-4694-9b90-7f13bdf25a96 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:07.334745 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b7972-50ee-4607-81d4-6f9131083c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:07.335363 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b7972-50ee-4607-81d4-6f9131083c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:07.335642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b7972-50ee-4607-81d4-6f9131083c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:07.335917 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b7972-50ee-4607-81d4-6f9131083c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:07.336562 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b7972-50ee-4607-81d4-6f9131083c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:07.338200 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b7972-50ee-4607-81d4-6f9131083c1b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:07.350467 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1481/2919] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:49:07.354846 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfe83a-f260-4a9d-9274-d6c283ddc398 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:07.361878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:07.366432 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfe83a-f260-4a9d-9274-d6c283ddc398 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:07.371109 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae74021-b26b-444c-aef5-a7f031299dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:07.375068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae74021-b26b-444c-aef5-a7f031299dba admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:07.375068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae74021-b26b-444c-aef5-a7f031299dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:07.375068 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae74021-b26b-444c-aef5-a7f031299dba admin admin] PATH_INFO: `/v3/projects/d69d7dd6cb534df891f6fd4c189c347b/users/71581dfcb4264353a80a4e15fa1f56ad/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:07.387894 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae74021-b26b-444c-aef5-a7f031299dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69d7dd6cb534df891f6fd4c189c347b, user_id=71581dfcb4264353a80a4e15fa1f56ad, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:07.390604 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae74021-b26b-444c-aef5-a7f031299dba admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:07.456217 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1439/2920] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:49:07 2022] PUT /identity/v3/projects/d69d7dd6cb534df891f6fd4c189c347b/users/71581dfcb4264353a80a4e15fa1f56ad/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:49:07.460529 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b7972-50ee-4607-81d4-6f9131083c1b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:07.470647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b7972-50ee-4607-81d4-6f9131083c1b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:07.527781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b620e89-1394-40ee-8ec8-6cd8cf2ecd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:07.528494 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b620e89-1394-40ee-8ec8-6cd8cf2ecd73 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:07.528841 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b620e89-1394-40ee-8ec8-6cd8cf2ecd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:07.529188 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b620e89-1394-40ee-8ec8-6cd8cf2ecd73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:07.529863 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b620e89-1394-40ee-8ec8-6cd8cf2ecd73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:07.531465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b620e89-1394-40ee-8ec8-6cd8cf2ecd73 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:07.542858 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1482/2921] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 1 05:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:49:07.548242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae74021-b26b-444c-aef5-a7f031299dba admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:07.562546 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae74021-b26b-444c-aef5-a7f031299dba admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:07.570673 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fd1c2-8d40-489e-b09f-93e1b29a3569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:07.575788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fd1c2-8d40-489e-b09f-93e1b29a3569 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:07.575788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fd1c2-8d40-489e-b09f-93e1b29a3569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:07.575788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fd1c2-8d40-489e-b09f-93e1b29a3569 admin admin] PATH_INFO: `/v3/projects/d69d7dd6cb534df891f6fd4c189c347b/users/71581dfcb4264353a80a4e15fa1f56ad/roles/4bd9fea9bc2648c2ada62148466f2700` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:07.575788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fd1c2-8d40-489e-b09f-93e1b29a3569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69d7dd6cb534df891f6fd4c189c347b, user_id=71581dfcb4264353a80a4e15fa1f56ad, role_id=4bd9fea9bc2648c2ada62148466f2700)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:07.576953 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fd1c2-8d40-489e-b09f-93e1b29a3569 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:07.592540 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1440/2922] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 1 05:49:07 2022] PUT /identity/v3/projects/d69d7dd6cb534df891f6fd4c189c347b/users/71581dfcb4264353a80a4e15fa1f56ad/roles/4bd9fea9bc2648c2ada62148466f2700 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:49:13.743780 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd9216-7cf0-44c5-b223-414e97e26836 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:13.744276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd9216-7cf0-44c5-b223-414e97e26836 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:13.744276 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd9216-7cf0-44c5-b223-414e97e26836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:13.780125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a2cd9216-7cf0-44c5-b223-414e97e26836 None None] MFA Rules not processed for user `71581dfcb4264353a80a4e15fa1f56ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:49:13.782136 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a2cd9216-7cf0-44c5-b223-414e97e26836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:13.840609 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:13.887558 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1483/2923] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:49:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:49:13.904412 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fd1c2-8d40-489e-b09f-93e1b29a3569 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:13.906654 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3b1fd1c2-8d40-489e-b09f-93e1b29a3569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:13.999566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fd1c2-8d40-489e-b09f-93e1b29a3569 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:14.014938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933eb6d9-d775-4289-9362-0a6661740601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:14.014938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933eb6d9-d775-4289-9362-0a6661740601 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:14.014938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933eb6d9-d775-4289-9362-0a6661740601 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:14.014938 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933eb6d9-d775-4289-9362-0a6661740601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:14.079563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933eb6d9-d775-4289-9362-0a6661740601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:14.081382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933eb6d9-d775-4289-9362-0a6661740601 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:14.092537 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1441/2924] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:49:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:49:20.173074 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d2985-3b0d-4b4e-954f-5c1c4e592a46 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:20.173781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d2985-3b0d-4b4e-954f-5c1c4e592a46 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:20.173781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d2985-3b0d-4b4e-954f-5c1c4e592a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:20.194744 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-904d2985-3b0d-4b4e-954f-5c1c4e592a46 None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:49:20.196514 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-904d2985-3b0d-4b4e-954f-5c1c4e592a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:20.259182 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1484/2925] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 1 05:49:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:49:20.391568 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933eb6d9-d775-4289-9362-0a6661740601 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:20.392999 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-933eb6d9-d775-4289-9362-0a6661740601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:20.467244 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933eb6d9-d775-4289-9362-0a6661740601 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:20.472433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619e97e-ed6e-4384-b9d7-15867af864b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:20.473034 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619e97e-ed6e-4384-b9d7-15867af864b3 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:20.473311 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619e97e-ed6e-4384-b9d7-15867af864b3 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:20.473615 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619e97e-ed6e-4384-b9d7-15867af864b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:20.479408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:20.483134 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619e97e-ed6e-4384-b9d7-15867af864b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:20.484717 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619e97e-ed6e-4384-b9d7-15867af864b3 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:20.500396 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1442/2926] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:49:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:49:30.251844 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=38 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:30.543631 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:43.333081 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad7c28-83b7-44a5-b34c-7558512768dc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:43.333584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad7c28-83b7-44a5-b34c-7558512768dc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:43.333709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad7c28-83b7-44a5-b34c-7558512768dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:43.362133 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-84ad7c28-83b7-44a5-b34c-7558512768dc None None] MFA Rules not processed for user `f58aa170e3004197a15dc2dc868d8681`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:49:43.363321 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-84ad7c28-83b7-44a5-b34c-7558512768dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:43.389709 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1485/2927] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 1 05:49:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:49:43.402494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619e97e-ed6e-4384-b9d7-15867af864b3 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:43.441985 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619e97e-ed6e-4384-b9d7-15867af864b3 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:43.447798 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7d4ae-1847-48eb-89ca-c846ab5b9ef7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b723681faf15434e8529e8fc56c6e9b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:43.448385 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7d4ae-1847-48eb-89ca-c846ab5b9ef7 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:43.448659 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7d4ae-1847-48eb-89ca-c846ab5b9ef7 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:43.448925 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7d4ae-1847-48eb-89ca-c846ab5b9ef7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:43.457235 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7d4ae-1847-48eb-89ca-c846ab5b9ef7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:43.458879 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7d4ae-1847-48eb-89ca-c846ab5b9ef7 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:43.466892 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1443/2928] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 1 05:49:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:49:46.265504 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b6d26-b085-456c-9d63-61a4ef83416f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:46.270511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b6d26-b085-456c-9d63-61a4ef83416f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:46.270511 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b6d26-b085-456c-9d63-61a4ef83416f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:46.285001 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c84b6d26-b085-456c-9d63-61a4ef83416f None None] MFA Rules not processed for user `99c0ab8ddd164b44a22370a7cc1042be`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:49:46.286234 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c84b6d26-b085-456c-9d63-61a4ef83416f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:46.293925 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1486/2929] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:49:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:49:46.334241 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7d4ae-1847-48eb-89ca-c846ab5b9ef7 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:46.336456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-abb7d4ae-1847-48eb-89ca-c846ab5b9ef7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:46.409050 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7d4ae-1847-48eb-89ca-c846ab5b9ef7 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:46.414744 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:46.415747 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:46.416022 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:46.416290 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] PATH_INFO: `/v3/users/46ecccda1e7b49969b26c6414c9c490a` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:46.417408 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46ecccda1e7b49969b26c6414c9c490a)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:46.419369 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:46.561199 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46ecccda1e7b49969b26c6414c9c490a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.561914 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46ecccda1e7b49969b26c6414c9c490a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.627160 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46ecccda1e7b49969b26c6414c9c490a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.641353 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46ecccda1e7b49969b26c6414c9c490a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.648942 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46ecccda1e7b49969b26c6414c9c490a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.655211 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1444/2930] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:49:46 2022] DELETE /identity/v3/users/46ecccda1e7b49969b26c6414c9c490a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:49:46.909062 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b6d26-b085-456c-9d63-61a4ef83416f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:46.922114 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b6d26-b085-456c-9d63-61a4ef83416f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:46.994971 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:46.995606 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:46.995886 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:46.996153 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] PATH_INFO: `/v3/projects/2f22ba19901d489db974b2f09cee6e0b` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:46.997280 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f22ba19901d489db974b2f09cee6e0b)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:46.999296 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:47.084411 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f22ba19901d489db974b2f09cee6e0b', 'request_id': 'req-bff26551-e420-4357-9049-f48ecdf2ffd5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:47.147000 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f22ba19901d489db974b2f09cee6e0b', 'request_id': 'req-bff26551-e420-4357-9049-f48ecdf2ffd5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:47.154278 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f22ba19901d489db974b2f09cee6e0b', 'request_id': 'req-bff26551-e420-4357-9049-f48ecdf2ffd5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:47.165934 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] The token cache is being invalidate because project 2f22ba19901d489db974b2f09cee6e0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:49:47.166214 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:47.166853 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:47.168337 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1487/2931] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:49:46 2022] DELETE /identity/v3/projects/2f22ba19901d489db974b2f09cee6e0b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:49:56.651937 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=73,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:57.162439 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:50:02.297712 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:50:02.299135 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:50:02.395431 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bdebd-0a75-41b9-9649-3c11402543d8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:50:02.457041 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cbb3c-80bd-453f-a149-d0b7effd5a59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:50:02.457670 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cbb3c-80bd-453f-a149-d0b7effd5a59 service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:50:02.457951 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cbb3c-80bd-453f-a149-d0b7effd5a59 service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:50:02.458218 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cbb3c-80bd-453f-a149-d0b7effd5a59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:50:02.460039 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ea7cbb3c-80bd-453f-a149-d0b7effd5a59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:50:02.472367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cbb3c-80bd-453f-a149-d0b7effd5a59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:50:02.474033 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cbb3c-80bd-453f-a149-d0b7effd5a59 service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:50:02.484691 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1445/2932] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:50:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:50:05.452783 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:50:05.454269 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:50:05.510948 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff26551-e420-4357-9049-f48ecdf2ffd5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:50:05.514811 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248313e-570b-40d1-8b2e-861a17070fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:50:05.515404 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248313e-570b-40d1-8b2e-861a17070fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:50:05.515682 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248313e-570b-40d1-8b2e-861a17070fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:50:05.515953 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248313e-570b-40d1-8b2e-861a17070fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:50:05.517732 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3248313e-570b-40d1-8b2e-861a17070fb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:50:05.582125 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248313e-570b-40d1-8b2e-861a17070fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:50:05.583652 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248313e-570b-40d1-8b2e-861a17070fb0 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:50:05.593814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1488/2933] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:50:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:50:12.480206 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=67 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:50:15.589426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=50 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:52:21.594351 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cbb3c-80bd-453f-a149-d0b7effd5a59 service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:52:21.614031 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cbb3c-80bd-453f-a149-d0b7effd5a59 service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:52:21.618404 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02265bae-6d61-4d4d-94ea-1fc98d30ae96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:52:21.618997 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265bae-6d61-4d4d-94ea-1fc98d30ae96 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:52:21.619298 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265bae-6d61-4d4d-94ea-1fc98d30ae96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:52:21.619571 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265bae-6d61-4d4d-94ea-1fc98d30ae96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:52:21.621358 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-02265bae-6d61-4d4d-94ea-1fc98d30ae96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:52:21.711701 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02265bae-6d61-4d4d-94ea-1fc98d30ae96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:52:21.713136 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02265bae-6d61-4d4d-94ea-1fc98d30ae96 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:52:21.733724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1446/2934] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:52:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:52:22.914787 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248313e-570b-40d1-8b2e-861a17070fb0 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:52:22.935776 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248313e-570b-40d1-8b2e-861a17070fb0 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:52:22.939377 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d9452-651a-4b5d-8bb4-c196c34356fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:52:22.939973 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d9452-651a-4b5d-8bb4-c196c34356fb service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:52:22.940252 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d9452-651a-4b5d-8bb4-c196c34356fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:52:22.940525 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d9452-651a-4b5d-8bb4-c196c34356fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:52:22.942267 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-520d9452-651a-4b5d-8bb4-c196c34356fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:52:22.956576 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d9452-651a-4b5d-8bb4-c196c34356fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:52:22.958021 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d9452-651a-4b5d-8bb4-c196c34356fb service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:52:22.969722 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1489/2935] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:52:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:52:26.504197 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d565ff-524c-4257-8d72-71a3c13beeab None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:52:26.504724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d565ff-524c-4257-8d72-71a3c13beeab None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:52:26.504724 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d565ff-524c-4257-8d72-71a3c13beeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:52:26.524816 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-90d565ff-524c-4257-8d72-71a3c13beeab None None] MFA Rules not processed for user `99c0ab8ddd164b44a22370a7cc1042be`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:52:26.526036 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-90d565ff-524c-4257-8d72-71a3c13beeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:52:26.581250 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1447/2936] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:52:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:52:32.964584 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=10 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:52:36.574509 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=75 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:54:06.876964 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d9452-651a-4b5d-8bb4-c196c34356fb service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:54:06.885692 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d9452-651a-4b5d-8bb4-c196c34356fb service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:54:06.889199 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef371b-b41e-4055-a067-5bff1d8d234d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:54:06.889812 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef371b-b41e-4055-a067-5bff1d8d234d service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:54:06.890086 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef371b-b41e-4055-a067-5bff1d8d234d service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:54:06.890353 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef371b-b41e-4055-a067-5bff1d8d234d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:54:06.892100 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-adef371b-b41e-4055-a067-5bff1d8d234d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:54:06.915672 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef371b-b41e-4055-a067-5bff1d8d234d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:54:06.917104 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef371b-b41e-4055-a067-5bff1d8d234d service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:54:06.924990 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1490/2937] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:54:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:54:16.921393 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=11 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:54:38.883702 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d565ff-524c-4257-8d72-71a3c13beeab None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:54:38.885213 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-90d565ff-524c-4257-8d72-71a3c13beeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:54:38.963489 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d565ff-524c-4257-8d72-71a3c13beeab None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:54:38.967130 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc39c52-f2cd-4ca7-ae0d-246d743fb57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0e8589931b46ca84168a2ad273b63d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:54:38.967731 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc39c52-f2cd-4ca7-ae0d-246d743fb57a service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:54:38.968008 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc39c52-f2cd-4ca7-ae0d-246d743fb57a service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:54:38.968278 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc39c52-f2cd-4ca7-ae0d-246d743fb57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:54:38.970061 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4dc39c52-f2cd-4ca7-ae0d-246d743fb57a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:54:38.981499 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc39c52-f2cd-4ca7-ae0d-246d743fb57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:54:38.982867 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc39c52-f2cd-4ca7-ae0d-246d743fb57a service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:54:38.991878 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1448/2938] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:54:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:54:48.987741 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=35 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:54:51.718426 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00aac8-d69a-4505-98a1-6ecbf26e247a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:54:51.718962 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00aac8-d69a-4505-98a1-6ecbf26e247a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:54:51.719090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00aac8-d69a-4505-98a1-6ecbf26e247a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:54:51.738581 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5a00aac8-d69a-4505-98a1-6ecbf26e247a None None] MFA Rules not processed for user `99c0ab8ddd164b44a22370a7cc1042be`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:54:51.739767 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5a00aac8-d69a-4505-98a1-6ecbf26e247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:54:51.746995 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1491/2939] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 1 05:54:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:54:52.068460 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc39c52-f2cd-4ca7-ae0d-246d743fb57a service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:54:52.078639 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc39c52-f2cd-4ca7-ae0d-246d743fb57a service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:54:52.082330 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1a4f3-77e2-44be-a504-2a309ebfc2d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:54:52.082917 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a4f3-77e2-44be-a504-2a309ebfc2d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:54:52.083195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a4f3-77e2-44be-a504-2a309ebfc2d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:54:52.083465 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a4f3-77e2-44be-a504-2a309ebfc2d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:54:52.085247 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-61b1a4f3-77e2-44be-a504-2a309ebfc2d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:54:52.095389 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1a4f3-77e2-44be-a504-2a309ebfc2d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:54:52.096850 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1a4f3-77e2-44be-a504-2a309ebfc2d3 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:54:52.104852 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1449/2940] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:54:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:54:52.848313 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00aac8-d69a-4505-98a1-6ecbf26e247a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:54:52.858697 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5a00aac8-d69a-4505-98a1-6ecbf26e247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:54:52.865419 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00aac8-d69a-4505-98a1-6ecbf26e247a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:54:52.871466 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cff5a-10e5-42ba-817d-10071bbd8470 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:54:52.872076 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cff5a-10e5-42ba-817d-10071bbd8470 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:54:52.872359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cff5a-10e5-42ba-817d-10071bbd8470 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:54:52.872633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cff5a-10e5-42ba-817d-10071bbd8470 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:54:52.874559 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0c8cff5a-10e5-42ba-817d-10071bbd8470 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:54:52.888371 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cff5a-10e5-42ba-817d-10071bbd8470 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:54:52.890015 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cff5a-10e5-42ba-817d-10071bbd8470 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:54:52.900981 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1492/2941] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:54:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:54:56.622692 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1a4f3-77e2-44be-a504-2a309ebfc2d3 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:54:56.624384 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-61b1a4f3-77e2-44be-a504-2a309ebfc2d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:54:56.641240 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1a4f3-77e2-44be-a504-2a309ebfc2d3 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:54:56.647201 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44932e-a74e-4dce-b31e-ebe3a8f39084 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:54:56.647898 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44932e-a74e-4dce-b31e-ebe3a8f39084 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:54:56.648242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44932e-a74e-4dce-b31e-ebe3a8f39084 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:54:56.648605 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44932e-a74e-4dce-b31e-ebe3a8f39084 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:54:56.650566 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7d44932e-a74e-4dce-b31e-ebe3a8f39084 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:54:56.664804 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44932e-a74e-4dce-b31e-ebe3a8f39084 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:54:56.666643 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44932e-a74e-4dce-b31e-ebe3a8f39084 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:54:56.679367 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1450/2942] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:54:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:55:02.896866 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:55:06.674541 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=12 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:55:07.166636 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cff5a-10e5-42ba-817d-10071bbd8470 service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:55:07.168399 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0c8cff5a-10e5-42ba-817d-10071bbd8470 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:55:07.181551 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cff5a-10e5-42ba-817d-10071bbd8470 service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:55:07.187523 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb2843-a3d2-4342-9cc1-134badace21d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:55:07.188154 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb2843-a3d2-4342-9cc1-134badace21d service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:55:07.188430 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb2843-a3d2-4342-9cc1-134badace21d service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:55:07.188695 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb2843-a3d2-4342-9cc1-134badace21d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:55:07.197781 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb2843-a3d2-4342-9cc1-134badace21d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:55:07.199980 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb2843-a3d2-4342-9cc1-134badace21d service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:55:07.210333 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1493/2943] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:55:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:55:09.344636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44932e-a74e-4dce-b31e-ebe3a8f39084 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:55:09.357841 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44932e-a74e-4dce-b31e-ebe3a8f39084 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:55:09.363360 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c74387-5225-4c76-9cc2-496b2403c34f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:55:09.363961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c74387-5225-4c76-9cc2-496b2403c34f service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:55:09.364242 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c74387-5225-4c76-9cc2-496b2403c34f service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:55:09.364519 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c74387-5225-4c76-9cc2-496b2403c34f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:55:09.366579 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-12c74387-5225-4c76-9cc2-496b2403c34f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:55:09.379382 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c74387-5225-4c76-9cc2-496b2403c34f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:55:09.381252 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c74387-5225-4c76-9cc2-496b2403c34f service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:55:09.392615 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1451/2944] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:55:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:55:13.155175 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb2843-a3d2-4342-9cc1-134badace21d service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:55:13.164704 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb2843-a3d2-4342-9cc1-134badace21d service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:55:13.168268 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933950-03b8-4109-a64b-85238b597892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:55:13.168856 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933950-03b8-4109-a64b-85238b597892 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:55:13.169135 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933950-03b8-4109-a64b-85238b597892 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:55:13.169408 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933950-03b8-4109-a64b-85238b597892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:55:13.178310 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79933950-03b8-4109-a64b-85238b597892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:55:13.179733 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79933950-03b8-4109-a64b-85238b597892 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:55:13.189684 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1494/2945] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:55:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:55:19.387919 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=6 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:55:23.185433 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=12 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:57:05.008189 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c74387-5225-4c76-9cc2-496b2403c34f service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:57:05.022146 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c74387-5225-4c76-9cc2-496b2403c34f service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:57:05.028961 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514124c-4b43-48e7-a86d-8f6043a27f03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cda3c1f67e54ee08abc014399b07cc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:57:05.029636 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514124c-4b43-48e7-a86d-8f6043a27f03 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:57:05.029915 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514124c-4b43-48e7-a86d-8f6043a27f03 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:57:05.030221 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514124c-4b43-48e7-a86d-8f6043a27f03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:57:05.034711 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5514124c-4b43-48e7-a86d-8f6043a27f03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:57:05.049665 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514124c-4b43-48e7-a86d-8f6043a27f03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:57:05.052524 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514124c-4b43-48e7-a86d-8f6043a27f03 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:57:05.067656 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1452/2946] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:57:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:57:15.062427 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=6 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:57:20.701331 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933950-03b8-4109-a64b-85238b597892 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:57:20.710026 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933950-03b8-4109-a64b-85238b597892 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:57:20.713623 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647a26c-c06e-40fe-bf9f-f052ed048d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e21910fca40479ca04e7f0418f7dd2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:57:20.714773 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647a26c-c06e-40fe-bf9f-f052ed048d50 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:57:20.715054 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647a26c-c06e-40fe-bf9f-f052ed048d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:57:20.715359 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647a26c-c06e-40fe-bf9f-f052ed048d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:57:20.717215 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a647a26c-c06e-40fe-bf9f-f052ed048d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:57:20.813647 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647a26c-c06e-40fe-bf9f-f052ed048d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:57:20.815399 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647a26c-c06e-40fe-bf9f-f052ed048d50 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:57:20.835295 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1495/2947] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:57:24.455324 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514124c-4b43-48e7-a86d-8f6043a27f03 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:57:24.467402 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514124c-4b43-48e7-a86d-8f6043a27f03 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:57:24.472627 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879f982-e22b-4ac0-bd7a-2a11bcb08cc6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:57:24.473494 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879f982-e22b-4ac0-bd7a-2a11bcb08cc6 service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:57:24.473995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879f982-e22b-4ac0-bd7a-2a11bcb08cc6 service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:57:24.474456 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879f982-e22b-4ac0-bd7a-2a11bcb08cc6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:57:24.476229 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d879f982-e22b-4ac0-bd7a-2a11bcb08cc6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:57:24.488697 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879f982-e22b-4ac0-bd7a-2a11bcb08cc6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:57:24.490307 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879f982-e22b-4ac0-bd7a-2a11bcb08cc6 service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:57:24.505512 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1453/2948] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 05:57:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:57:28.246090 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647a26c-c06e-40fe-bf9f-f052ed048d50 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:57:28.249594 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a647a26c-c06e-40fe-bf9f-f052ed048d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:57:28.260148 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647a26c-c06e-40fe-bf9f-f052ed048d50 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:57:28.266002 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:57:28.266633 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:57:28.266913 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:57:28.267180 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] PATH_INFO: `/v3/users/99c0ab8ddd164b44a22370a7cc1042be` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:57:28.268299 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c0ab8ddd164b44a22370a7cc1042be)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:57:28.270256 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:57:28.523102 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c0ab8ddd164b44a22370a7cc1042be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:28.534706 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c0ab8ddd164b44a22370a7cc1042be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:28.541642 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c0ab8ddd164b44a22370a7cc1042be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:28.627254 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c0ab8ddd164b44a22370a7cc1042be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:28.627880 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c0ab8ddd164b44a22370a7cc1042be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:28.634536 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1496/2949] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:57:28 2022] DELETE /identity/v3/users/99c0ab8ddd164b44a22370a7cc1042be => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:57:28.878573 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879f982-e22b-4ac0-bd7a-2a11bcb08cc6 service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:57:28.888426 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879f982-e22b-4ac0-bd7a-2a11bcb08cc6 service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:57:28.948484 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:57:28.949280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:57:28.949602 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:57:28.949881 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] PATH_INFO: `/v3/projects/8929d2f4d729433bb807e9f0890c9b1f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:57:28.951037 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8929d2f4d729433bb807e9f0890c9b1f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:57:28.952995 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:57:29.052195 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8929d2f4d729433bb807e9f0890c9b1f', 'request_id': 'req-120474b8-9c47-4483-9813-f47ca3c53320'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:29.120561 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8929d2f4d729433bb807e9f0890c9b1f', 'request_id': 'req-120474b8-9c47-4483-9813-f47ca3c53320'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:29.127416 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8929d2f4d729433bb807e9f0890c9b1f', 'request_id': 'req-120474b8-9c47-4483-9813-f47ca3c53320'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:29.138433 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] The token cache is being invalidate because project 8929d2f4d729433bb807e9f0890c9b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:57:29.141998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:29.141998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:29.141998 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1454/2950] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:57:28 2022] DELETE /identity/v3/projects/8929d2f4d729433bb807e9f0890c9b1f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:57:29.144801 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:57:29.146231 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:57:29.160210 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24685c7b-f0ea-4be8-bd4f-a8a3a0a1efd9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:57:29.193410 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:57:29.217568 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:57:29.218191 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:57:29.218465 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:57:29.218731 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] PATH_INFO: `/v3/users/71581dfcb4264353a80a4e15fa1f56ad` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:57:29.219898 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71581dfcb4264353a80a4e15fa1f56ad)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:57:29.221846 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:57:29.431888 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71581dfcb4264353a80a4e15fa1f56ad'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:29.444714 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71581dfcb4264353a80a4e15fa1f56ad'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:29.452405 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71581dfcb4264353a80a4e15fa1f56ad'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:29.543604 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71581dfcb4264353a80a4e15fa1f56ad'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:29.544223 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71581dfcb4264353a80a4e15fa1f56ad'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:29.550492 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1497/2951] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 1 05:57:29 2022] DELETE /identity/v3/users/71581dfcb4264353a80a4e15fa1f56ad => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:57:29.800587 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:57:29.802144 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:57:29.818563 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120474b8-9c47-4483-9813-f47ca3c53320 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:57:29.878527 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d4d38ef7ecd40a380aba27a2b564d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61f9dee9572644e580e35da89b6265e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:57:29.879172 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:57:29.879756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:57:29.879756 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] PATH_INFO: `/v3/projects/d69d7dd6cb534df891f6fd4c189c347b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:57:29.880813 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69d7dd6cb534df891f6fd4c189c347b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:57:29.882740 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:57:29.905280 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:57:30.007693 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69d7dd6cb534df891f6fd4c189c347b', 'request_id': 'req-d67849f9-ab99-476c-a22f-f9c59c392ae7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:30.092053 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69d7dd6cb534df891f6fd4c189c347b', 'request_id': 'req-d67849f9-ab99-476c-a22f-f9c59c392ae7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:30.098788 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69d7dd6cb534df891f6fd4c189c347b', 'request_id': 'req-d67849f9-ab99-476c-a22f-f9c59c392ae7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:30.110596 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] The token cache is being invalidate because project d69d7dd6cb534df891f6fd4c189c347b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:57:30.110886 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:30.111567 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d67849f9-ab99-476c-a22f-f9c59c392ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:57:30.113047 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1455/2952] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 1 05:57:29 2022] DELETE /identity/v3/projects/d69d7dd6cb534df891f6fd4c189c347b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:57:39.546759 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:57:40.108476 dv-cinder-852208-15-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 06:00:11.136689 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 06:00:11.138124 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 06:00:11.294728 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf6f3f-fef9-4de7-82fa-b7ce071e4949 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 06:00:11.352900 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764201f-1f76-4d82-97fc-667ebb78c9cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c11c7fb612ab40d88f526b8b38dbfb23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4b03d9fda4f49cb92fe52745e17e5bf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 06:00:11.353531 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764201f-1f76-4d82-97fc-667ebb78c9cc service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 06:00:11.353814 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764201f-1f76-4d82-97fc-667ebb78c9cc service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 06:00:11.354088 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764201f-1f76-4d82-97fc-667ebb78c9cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 06:00:11.355848 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1764201f-1f76-4d82-97fc-667ebb78c9cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 06:00:11.366961 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764201f-1f76-4d82-97fc-667ebb78c9cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 06:00:11.368513 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764201f-1f76-4d82-97fc-667ebb78c9cc service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 06:00:11.378442 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1498/2953] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 1 06:00:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 06:00:21.374045 dv-cinder-852208-15-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}